-- Logs begin at Tue 2022-07-12 06:32:17 UTC, end at Tue 2022-07-12 07:33:35 UTC. -- Jul 12 06:58:36.734507 dv-cinder-825492-25-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 06:58:36.742065 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 06:58:36 2022] *** Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: nodename: dv-cinder-825492-25-iscsi Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: machine: x86_64 Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: clock source: unix Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: pcre jit disabled Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: detected number of CPU cores: 4 Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: current working directory: / Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: detected binary path: /usr/bin/uwsgi-core Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: your processes number limit is 47680 Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: your memory page size is 4096 bytes Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: detected max file descriptor number: 2048 Jul 12 06:58:36.744102 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: lock engine: pthread robust mutexes Jul 12 06:58:36.744976 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: thunder lock: enabled Jul 12 06:58:36.744976 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 06:58:36.744976 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 06:58:36.789893 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: Python main interpreter initialized at 0x55be82be29c0 Jul 12 06:58:36.789893 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: python threads support enabled Jul 12 06:58:36.789893 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: your server socket listen backlog is limited to 100 connections Jul 12 06:58:36.789893 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: your mercy for graceful operations on workers is 90 seconds Jul 12 06:58:36.790277 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 06:58:36.790348 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: *** Operational MODE: preforking *** Jul 12 06:58:36.790612 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: *** uWSGI is running in multiple interpreter mode *** Jul 12 06:58:36.790612 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: spawned uWSGI master process (pid: 80462) Jul 12 06:58:36.790832 dv-cinder-825492-25-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 06:58:36.791336 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: spawned uWSGI worker 1 (pid: 80472, cores: 1) Jul 12 06:58:36.791667 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: spawned uWSGI worker 2 (pid: 80473, cores: 1) Jul 12 06:58:36.791834 dv-cinder-825492-25-iscsi devstack@keystone.service[80462]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 06:58:42.159234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 06:58:42.159234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: from cryptography.utils import int_from_bytes Jul 12 06:58:42.160068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 06:58:42.160068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: from cryptography.utils import int_from_bytes Jul 12 06:58:42.160493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 06:58:42.160493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: from cryptography.utils import int_from_bytes Jul 12 06:58:42.160615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 06:58:42.160615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: from cryptography.utils import int_from_bytes Jul 12 06:58:42.208108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 06:58:42.208299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 06:58:42.208522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] command line args: [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 06:58:42.208809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 06:58:42.208939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 06:58:42.209031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ================================================================================ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 06:58:42.209140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 06:58:42.209278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] admin_token = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.209359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] command line args: [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 06:58:42.209560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config_dir = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.209684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 06:58:42.209684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ================================================================================ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 06:58:42.209830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.209962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] admin_token = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] config_source = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] debug = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_dir = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_source = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] debug = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.210916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.211946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_config_append = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_config_append = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_dir = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_dir = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.212947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.213977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.214778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.215981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] publish_errors = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] publish_errors = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.216979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.217921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] standard_threads = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] standard_threads = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_journal = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_json = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_journal = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.218942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_stderr = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.219012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_json = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.219126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] use_syslog = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.219224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_stderr = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.219298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_syslog = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.219367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.219561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 06:58:42.219695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.219787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.219864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.220980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.221952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222157 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.222871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.223900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.224973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.225987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.226914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.227967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.228904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.229943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.230061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.230175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.230411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.230985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.231954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232785 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.232943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.233966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.234869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.235974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.236888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.237879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.238936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239479 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.239942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.240988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.241987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.242939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.external = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.external = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.243865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.password = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] auth.token = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.password = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.244951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.token = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.245935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.246980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.247991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.248988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.249995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.250931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.251952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.252102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.252179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.252351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.252488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.252625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.252754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.252889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.253016 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.253173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.253303 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.253432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.253561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.253772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.253901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.254948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.255996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.256126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.256258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.256381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.256547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.256673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.256805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.256929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.257126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.257253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.257380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.257507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.257663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.257790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.257916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.258932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.259967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.260983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.261097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.261281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.261407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.261616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.261815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.262161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.262290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.262476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.262634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.262800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.262953 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.263103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.263252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.263402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.263562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.263722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.263872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.264022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.264169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.264326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.264476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.264633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.264783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.264934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.265114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.265262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.265417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.265572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.265727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.265873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.266020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.266165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.266309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.266464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.266608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.266753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.266896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.267042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.267189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.267342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.267488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.267488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.267776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.267923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.268067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.268221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.268366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.268509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.268652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.268797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.268951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.269126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.269277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.269426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.269575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.269736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.269896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.270046 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.270194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.270343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.270490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.270636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.270797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.270945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.271093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.271242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.271390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.271537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.271693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.271852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.272001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.272153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.272308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.272456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.272615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.272761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.272910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.273120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.273269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.273415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.273571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.273716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.273878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.274889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.275160 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.275320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.275467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.275611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.275756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.275910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.276054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.276054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.276336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.276481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.276623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.276766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.276910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.277090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.277256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.277427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.277565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.277678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.277894 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.278064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.278212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.278359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.278506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.278654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.278800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.278955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.279104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.279251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.279400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.279547 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.279696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.279854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.280004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.280162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.280309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.280454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.280601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.280762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.280909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.281093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.281239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.281384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.281527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.281755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.281901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.282045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.282207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.282350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.282494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.282646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.282804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.282952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.283098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.283242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.283387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.283541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.283687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.283830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.283974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.284118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.284276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.284430 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.284574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.284718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.284864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.285008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.285193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.285343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.285490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.285490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.285770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.285920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.286078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.286239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.286388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.286536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.286683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.286832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.286989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.287137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.287285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.287432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.287581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.287729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.287887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.288036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.288195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.288342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.288490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.288637 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.288794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.288942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.289158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.289303 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.289447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.289594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.289751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.289751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.290921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.291194 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.291339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.291493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.291493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.291766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.291910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.292051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.292220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.292365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.292509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.292659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.292871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.292871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.293179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.293179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.293462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.293610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.293759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.293908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.294972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.295254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.295401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.295550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.295697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.295853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.296002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.296150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.296307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.296456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.296603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.296759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.296909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.297077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.297077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.297369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.297518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.297675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.297833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.297995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.298143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.298290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.298447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.298597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.298597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.298887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.299033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.299179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.299335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.299493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.299640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.299787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.299942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.300089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.300236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.300382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.300538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.300686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.300842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.300992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.301153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.301298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.301444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.301587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.301742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.301887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.302032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.302181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.302326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.302471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.302632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.302778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.302923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.303069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.303213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80473) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 06:58:42.303361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.303361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.303361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.303361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.303361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 06:58:42.303361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 06:58:42.322767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.322994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.323092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.323235 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.323357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.323665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.332081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.332291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.360335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.360488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.360647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.360853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.506358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.506992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.554545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.554764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.555505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.555727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.618460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.618711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.618887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.618980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.619116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.619202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.619296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.619385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.619534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.619675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.619814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.620018 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.644422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.644578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.644764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.644901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.645956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.646106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.657233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.657358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.669992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.670264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.670433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.670525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.670655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.670783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.670911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.671045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.671178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.671315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.671479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80473) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.671606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.671733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.671868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 06:58:42.688505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.688707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 06:58:42.689079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.689275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.689393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 06:58:42.689534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 06:58:42.689680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.689764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.689886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 06:58:42.689970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.690108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.690195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.690318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.690402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.690523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.690789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.701548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.701790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.702077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.702352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.702478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.702696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.702847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.703154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.710534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.711453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.713781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.714060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.714308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.714607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.714940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.715103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.715190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.715255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.715329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.715528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.715669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.715812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.715895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.716043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.716170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.716291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.716399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.716575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.716704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.716891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.717186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.717450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.742967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.743838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.744542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.744815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.745482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.745762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.750221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.750493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.750767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.751057 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.751350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.751739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.758179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.758398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.759008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.759285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.762954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.763237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.763508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.763780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.763976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.764073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.764205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.764351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.764477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.764603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.764738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.764910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.765038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.765189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.765355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.765651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.783284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.783554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.784627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.784902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.788062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.789437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.791676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.791966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.793162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.793490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.797105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.798588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.800672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.802374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.804335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.804633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.804916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.805233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.805505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.805775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.805985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.806298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.806591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.806873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.807167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.807438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.825328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.825597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.825860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.826119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.827228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.827511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.827790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.828054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.833990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.835678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.835943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.836109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.837662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.837938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.839105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.839383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.839662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.839974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.840257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.841276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.841555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.841834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.842107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.842415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.853453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.853728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.854005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.854289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.854563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.854851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.855116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.855653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.855930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.856210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.856466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.856741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.858322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.858614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.874175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.876567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.877423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.880018 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.880688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.882309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.883334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.884040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.884312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.884586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.885072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.886758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.887004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.887312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.891867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.893624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.894682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.896297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.896821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.897099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.897409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.897691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.899532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.899818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.900202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.900480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.906947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.907225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.907514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.909727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.910012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.910289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.915292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80473) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.915566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.915842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.916111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.916374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.916650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.916919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.917214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.917582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.917935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.918172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.918272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 06:58:42.918467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.918611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.918737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.918892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.919034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80473) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.919211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.919442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.919826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.920106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.920386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.920659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.920937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.921247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.921520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.921784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.922049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 06:58:42.956916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55be82be29c0 pid: 80473 (default app) Jul 12 06:58:42.960098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55be82be29c0 pid: 80472 (default app) Jul 12 06:58:42.960265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077c764-b2da-4910-b966-ec65e81c4f08 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:42.960571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077c764-b2da-4910-b966-ec65e81c4f08 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:42.960848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077c764-b2da-4910-b966-ec65e81c4f08 None None] PATH_INFO: `/v3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:42.965354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Tue Jul 12 06:58:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:58:50.698997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe756-bac5-4e10-83a4-bde191a96a4d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:50.699898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe756-bac5-4e10-83a4-bde191a96a4d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:50.699898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe756-bac5-4e10-83a4-bde191a96a4d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:50.706618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:58:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:58:50.715136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:50.716069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:50.716069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:50.784881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: INFO dbcounter [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] Registered counter for database keystone Jul 12 06:58:50.810466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG oslo_db.sqlalchemy.engines [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80473) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 06:58:50.822079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writer thread running {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 06:58:51.009638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG passlib.handlers.bcrypt [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80473) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 06:58:51.010046 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG passlib.handlers.bcrypt [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80473) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 06:58:51.065504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:58:51.067106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:58:51.208981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:58:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 06:58:51.462500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe756-bac5-4e10-83a4-bde191a96a4d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:51.532403 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: INFO dbcounter [None req-48dbe756-bac5-4e10-83a4-bde191a96a4d None None] Registered counter for database keystone Jul 12 06:58:51.554182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG oslo_db.sqlalchemy.engines [None req-48dbe756-bac5-4e10-83a4-bde191a96a4d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80472) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 06:58:51.564367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writer thread running {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 06:58:51.684558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe756-bac5-4e10-83a4-bde191a96a4d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:51.690109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999201a-6b79-42ad-b1e6-888a9ed08e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:51.690787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999201a-6b79-42ad-b1e6-888a9ed08e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:51.691078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999201a-6b79-42ad-b1e6-888a9ed08e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:51.691358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999201a-6b79-42ad-b1e6-888a9ed08e41 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:51.701134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999201a-6b79-42ad-b1e6-888a9ed08e41 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:51.752391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1999201a-6b79-42ad-b1e6-888a9ed08e41 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 06:58:51.752664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999201a-6b79-42ad-b1e6-888a9ed08e41 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:51.764223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Jul 12 06:58:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 06:58:51.773250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:51.790907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c620f6d-0311-45cf-928f-7eb518767fc0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:51.796951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544cfcc-8c69-432f-856d-21a5d13629d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:51.798148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544cfcc-8c69-432f-856d-21a5d13629d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:51.798439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544cfcc-8c69-432f-856d-21a5d13629d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:51.798711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544cfcc-8c69-432f-856d-21a5d13629d5 admin admin] PATH_INFO: `/v3/projects/91c604937dd14031814263870ccac629` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:51.800346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544cfcc-8c69-432f-856d-21a5d13629d5 admin admin] RBAC: Authorizing `identity:get_project(project_id=91c604937dd14031814263870ccac629)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:51.854270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2544cfcc-8c69-432f-856d-21a5d13629d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 06:58:51.854611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544cfcc-8c69-432f-856d-21a5d13629d5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:51.859401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Tue Jul 12 06:58:51 2022] GET /identity/v3/projects/91c604937dd14031814263870ccac629 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:58:53.876499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701e7f4-785d-446a-9a9e-5bc71e000ddd None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:53.877403 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701e7f4-785d-446a-9a9e-5bc71e000ddd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:53.877403 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701e7f4-785d-446a-9a9e-5bc71e000ddd None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:53.884126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:58:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:58:53.890855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe5e3f-8135-4490-bb5e-30e487612c73 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:53.891564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe5e3f-8135-4490-bb5e-30e487612c73 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:53.891711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe5e3f-8135-4490-bb5e-30e487612c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:53.915315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3bbe5e3f-8135-4490-bb5e-30e487612c73 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:58:53.916838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3bbe5e3f-8135-4490-bb5e-30e487612c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:58:53.925321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:58:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 06:58:53.948576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701e7f4-785d-446a-9a9e-5bc71e000ddd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:53.968445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701e7f4-785d-446a-9a9e-5bc71e000ddd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:53.973747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d04b3-5ed2-4575-8aea-c04f2e46381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:53.974389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d04b3-5ed2-4575-8aea-c04f2e46381f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:53.974676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d04b3-5ed2-4575-8aea-c04f2e46381f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:53.974957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d04b3-5ed2-4575-8aea-c04f2e46381f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:53.983012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d04b3-5ed2-4575-8aea-c04f2e46381f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:53.984799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d04b3-5ed2-4575-8aea-c04f2e46381f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:53.994037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Jul 12 06:58:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 06:58:53.999784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe5e3f-8135-4490-bb5e-30e487612c73 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:54.009189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe5e3f-8135-4490-bb5e-30e487612c73 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:54.014022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbf6fc-5d28-4797-925d-44160e3a487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:54.014897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbf6fc-5d28-4797-925d-44160e3a487a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:54.015179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbf6fc-5d28-4797-925d-44160e3a487a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:54.015453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbf6fc-5d28-4797-925d-44160e3a487a admin admin] PATH_INFO: `/v3/users/3921c7a7e4fd41ef9eea262b7a97405a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:54.016894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbf6fc-5d28-4797-925d-44160e3a487a admin admin] RBAC: Authorizing `identity:get_user(user_id=3921c7a7e4fd41ef9eea262b7a97405a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:54.018753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbf6fc-5d28-4797-925d-44160e3a487a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:54.020325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Tue Jul 12 06:58:53 2022] GET /identity/v3/users/3921c7a7e4fd41ef9eea262b7a97405a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:58:56.081177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca381c2-44df-4977-86cc-af35050b859d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.081744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca381c2-44df-4977-86cc-af35050b859d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.081744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca381c2-44df-4977-86cc-af35050b859d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.083088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:58:56.088406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e7ee8-37b1-46df-9ae3-7a1303552230 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.088820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e7ee8-37b1-46df-9ae3-7a1303552230 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.088968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e7ee8-37b1-46df-9ae3-7a1303552230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.108078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b19e7ee8-37b1-46df-9ae3-7a1303552230 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:58:56.109320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b19e7ee8-37b1-46df-9ae3-7a1303552230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:58:56.141349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e91f8-ef83-4a05-ae72-37ca886a52f0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.141611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e91f8-ef83-4a05-ae72-37ca886a52f0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.141889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e91f8-ef83-4a05-ae72-37ca886a52f0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.143263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:58:56.148416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.148733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.149010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.193623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG passlib.handlers.bcrypt [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80472) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 06:58:56.194032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG passlib.handlers.bcrypt [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80472) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 06:58:56.197143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:58:56.225288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e7ee8-37b1-46df-9ae3-7a1303552230 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.242042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:58:56.243596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:58:56.252366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:58:56.337231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e7ee8-37b1-46df-9ae3-7a1303552230 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:56.345195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:58:56.367226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.370438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:56.371248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.371535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.371817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.383212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:58:56.394477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:58:56.395393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:58:56.396226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:58:56.401520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.471241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe04c4-be03-4b72-bfec-49f971ceb329 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:56.473740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:58:56.483471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:58:56.502396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac35282-5e8a-4647-95dc-cd0c9fa1eedb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:56.509945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:56.510860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.511149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.511434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.541352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:56.542026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.542308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.542560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.543335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:58:56.543782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:56.545225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:56.557233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:58:56.563109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.592317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:56.593897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:56.672405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14145b3d-382b-44ee-8d73-e8a8f56eca29 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:56.696497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:58:56.702620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.706177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:58:56.711490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:56.712384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.712665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.712940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] PATH_INFO: `/v3/users/3921c7a7e4fd41ef9eea262b7a97405a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.714178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] RBAC: Authorizing `identity:get_user(user_id=3921c7a7e4fd41ef9eea262b7a97405a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:56.716138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:56.718142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/users/3921c7a7e4fd41ef9eea262b7a97405a => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:58:56.723420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.738481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:58:56.804225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0990a-2696-4a93-964f-dfecec91ca48 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:56.827919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58122a3-2056-4539-8beb-3b1a15db20c2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:56.839672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:56.840819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.841228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.841514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.860499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:56.862026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:56.863884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:56.864682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:56.864963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:56.865268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:56.866481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:56.867920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:56.869566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:58:56.875295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 06:58:56.883175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:58:56.884010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:58:56.885103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 06:58:56.885103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 06:58:56.885103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 06:58:56.885103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:58:56.885103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:58:56.890786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:56.947415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:58:56.963990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:58:56.983556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d523a8-3dfd-4b2a-84a5-b8bd7a310b13 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:56.993431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468c7b6-2c0c-445e-8c5a-24aa4372e2dd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:57.019354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e5f64-9794-4d82-90ed-5cf01c8aa5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:57.020027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e5f64-9794-4d82-90ed-5cf01c8aa5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:57.020314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e5f64-9794-4d82-90ed-5cf01c8aa5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:57.020592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e5f64-9794-4d82-90ed-5cf01c8aa5e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:57.021476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e5f64-9794-4d82-90ed-5cf01c8aa5e1 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.domain.id=default, user.id=3921c7a7e4fd41ef9eea262b7a97405a) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:58:57.021794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e5f64-9794-4d82-90ed-5cf01c8aa5e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:57.023249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e5f64-9794-4d82-90ed-5cf01c8aa5e1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:57.029726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a4388-7fd4-4ace-9cce-8677d9ffe527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:57.030333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/role_assignments?user.id=3921c7a7e4fd41ef9eea262b7a97405a&scope.domain.id=default&role.id=aaca343d2e4a43c88674834f08499f15 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:58:57.030458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4388-7fd4-4ace-9cce-8677d9ffe527 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:57.030671 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4388-7fd4-4ace-9cce-8677d9ffe527 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:57.030947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4388-7fd4-4ace-9cce-8677d9ffe527 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:57.031679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a4388-7fd4-4ace-9cce-8677d9ffe527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:58:57.031989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a4388-7fd4-4ace-9cce-8677d9ffe527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:57.033439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a4388-7fd4-4ace-9cce-8677d9ffe527 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:57.054097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 06:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:58:59.409740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9da673-d5d6-4392-88bc-22153e481cec None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:59.410730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9da673-d5d6-4392-88bc-22153e481cec None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:59.411305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9da673-d5d6-4392-88bc-22153e481cec None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:59.412888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:58:59.418078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12187-0233-417c-9c65-4d27a4f77c8c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:59.418470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12187-0233-417c-9c65-4d27a4f77c8c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:59.418784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d12187-0233-417c-9c65-4d27a4f77c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:59.442099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d0d12187-0233-417c-9c65-4d27a4f77c8c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:58:59.443316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d0d12187-0233-417c-9c65-4d27a4f77c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:58:59.538539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 11/23] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:58:59.573463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9da673-d5d6-4392-88bc-22153e481cec None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:58:59.737563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9da673-d5d6-4392-88bc-22153e481cec None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:58:59.785813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032f09b-504e-4703-8f6e-a08103bc7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:58:59.786488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032f09b-504e-4703-8f6e-a08103bc7462 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:59.786766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032f09b-504e-4703-8f6e-a08103bc7462 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:59.787036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032f09b-504e-4703-8f6e-a08103bc7462 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:59.787790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032f09b-504e-4703-8f6e-a08103bc7462 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:58:59.789228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032f09b-504e-4703-8f6e-a08103bc7462 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:58:59.855376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3504d-e71f-4506-95c0-3e9290d6e064 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:59.855960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3504d-e71f-4506-95c0-3e9290d6e064 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:59.857015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3504d-e71f-4506-95c0-3e9290d6e064 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:59.858581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:58:59.868992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e1dc-b1d9-4602-a2e3-5342af5b8d64 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:59.869499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e1dc-b1d9-4602-a2e3-5342af5b8d64 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:59.869919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e1dc-b1d9-4602-a2e3-5342af5b8d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:59.903687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9433e1dc-b1d9-4602-a2e3-5342af5b8d64 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:58:59.905181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9433e1dc-b1d9-4602-a2e3-5342af5b8d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:58:59.946686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 13/25] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 06:58:59 2022] POST /identity/v3/roles => generated 219 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:58:59.968693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54e0e-d461-4002-a7b8-953becb296f8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:59.969230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54e0e-d461-4002-a7b8-953becb296f8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:59.969643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54e0e-d461-4002-a7b8-953becb296f8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:59.971360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 14/26] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:58:59.977266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af1c15-7ff6-4aa7-9ec1-00dfa869ff46 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:58:59.977963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af1c15-7ff6-4aa7-9ec1-00dfa869ff46 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:58:59.978393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af1c15-7ff6-4aa7-9ec1-00dfa869ff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:58:59.994400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:00.012985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:00.022392 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-62af1c15-7ff6-4aa7-9ec1-00dfa869ff46 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:00.023829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-62af1c15-7ff6-4aa7-9ec1-00dfa869ff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:00.085766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 13/27] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:00.124214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e1dc-b1d9-4602-a2e3-5342af5b8d64 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:00.135756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 15/28] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:00.176358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af1c15-7ff6-4aa7-9ec1-00dfa869ff46 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:00.251542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e1dc-b1d9-4602-a2e3-5342af5b8d64 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:00.289639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899fc4f-d1ca-4659-8902-6ccd0ab1aa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:00.290686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899fc4f-d1ca-4659-8902-6ccd0ab1aa1a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.291085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899fc4f-d1ca-4659-8902-6ccd0ab1aa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.291456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899fc4f-d1ca-4659-8902-6ccd0ab1aa1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.299839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af1c15-7ff6-4aa7-9ec1-00dfa869ff46 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:00.305797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899fc4f-d1ca-4659-8902-6ccd0ab1aa1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:00.307474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899fc4f-d1ca-4659-8902-6ccd0ab1aa1a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6899fc4f-d1ca-4659-8902-6ccd0ab1aa1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 06:59:00.335312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:00.336343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:00.337364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 06:59:00.337364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 06:59:00.337364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 06:59:00.337364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:00.338939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 14/29] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:00.340772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd3f4-eb84-4d0d-b8dc-55efd579c35c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.341280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd3f4-eb84-4d0d-b8dc-55efd579c35c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.341843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd3f4-eb84-4d0d-b8dc-55efd579c35c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.344858 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 15/30] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:00.345843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd3f4-eb84-4d0d-b8dc-55efd579c35c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:00.354904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46999c69-ba83-4a51-9556-0c59eb3b29ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:00.355998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999c69-ba83-4a51-9556-0c59eb3b29ae None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.357284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999c69-ba83-4a51-9556-0c59eb3b29ae None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.357707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999c69-ba83-4a51-9556-0c59eb3b29ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.360444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:00.380375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46999c69-ba83-4a51-9556-0c59eb3b29ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:00.386191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-46999c69-ba83-4a51-9556-0c59eb3b29ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:00.389753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:00.390590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:00.391347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:00.393440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 16/31] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:00.395229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f36a2-4306-4a5c-ba12-6bc4513f4821 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.395655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f36a2-4306-4a5c-ba12-6bc4513f4821 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.396087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f36a2-4306-4a5c-ba12-6bc4513f4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.415289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-661f36a2-4306-4a5c-ba12-6bc4513f4821 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:00.417259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-661f36a2-4306-4a5c-ba12-6bc4513f4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:00.473614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd3f4-eb84-4d0d-b8dc-55efd579c35c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:00.493116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 17/32] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:00.493116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f36a2-4306-4a5c-ba12-6bc4513f4821 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:00.515248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766777d-863c-4e37-91e3-6b07e08e972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:00.515848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766777d-863c-4e37-91e3-6b07e08e972f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.516133 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766777d-863c-4e37-91e3-6b07e08e972f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.516407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766777d-863c-4e37-91e3-6b07e08e972f None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.517186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766777d-863c-4e37-91e3-6b07e08e972f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:00.517497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766777d-863c-4e37-91e3-6b07e08e972f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:00.518885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766777d-863c-4e37-91e3-6b07e08e972f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:00.530261 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 16/33] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 06:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:00.531876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10c403-f3b6-4b57-8b3e-543ba92879f5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.532371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10c403-f3b6-4b57-8b3e-543ba92879f5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.532835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10c403-f3b6-4b57-8b3e-543ba92879f5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.534327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 17/34] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:00.535178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10c403-f3b6-4b57-8b3e-543ba92879f5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:00.610611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f36a2-4306-4a5c-ba12-6bc4513f4821 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:00.636125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:00.636703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:00.648888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:00.649450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.649870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.650010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.650788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:00.651102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:00.652627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:00.652872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10c403-f3b6-4b57-8b3e-543ba92879f5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:00.661586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 18/35] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:00.662158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:00.696070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215a98-0274-4271-af85-caedd9583584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:00.696752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215a98-0274-4271-af85-caedd9583584 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.697036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215a98-0274-4271-af85-caedd9583584 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.697325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215a98-0274-4271-af85-caedd9583584 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.698462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215a98-0274-4271-af85-caedd9583584 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:00.699877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215a98-0274-4271-af85-caedd9583584 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:00.770058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa84c1-01aa-4bc5-9228-d21e4f1dde07 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:00.805772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:00.806447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.806738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.807019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.807733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:00.809348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:00.825089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 18/36] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 06:59:00 2022] POST /identity/v3/roles => generated 225 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:00.825089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78595178-6871-4aca-bb00-23bf5c06fcd1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:00.825089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78595178-6871-4aca-bb00-23bf5c06fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:00.825089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78595178-6871-4aca-bb00-23bf5c06fcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:00.840349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-78595178-6871-4aca-bb00-23bf5c06fcd1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:00.842382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-78595178-6871-4aca-bb00-23bf5c06fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:00.869571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 19/37] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 06:59:00 2022] POST /identity/v3/projects => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:00.870705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:00.929205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 19/38] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:00.969110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:00.999594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78595178-6871-4aca-bb00-23bf5c06fcd1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:01.097653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c4d1e-1738-4348-96f7-b4b98f22ce96 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:01.114294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:01.142245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78595178-6871-4aca-bb00-23bf5c06fcd1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:01.196327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721b074-b86d-4239-92ae-fef1f791ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:01.197041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b074-b86d-4239-92ae-fef1f791ffd7 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.197363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b074-b86d-4239-92ae-fef1f791ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.197993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b074-b86d-4239-92ae-fef1f791ffd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.198775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721b074-b86d-4239-92ae-fef1f791ffd7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:01.200394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721b074-b86d-4239-92ae-fef1f791ffd7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:01.219753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 20/39] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 06:59:00 2022] POST /identity/v3/roles => generated 223 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 06:59:01.224368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c4bb6-9557-49c5-8630-2897cbb82173 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.224651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c4bb6-9557-49c5-8630-2897cbb82173 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.224933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c4bb6-9557-49c5-8630-2897cbb82173 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.226676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 21/40] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:01.233393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c471eb5-d6f9-49f1-a3ab-3ee652cb3989 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.233740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c471eb5-d6f9-49f1-a3ab-3ee652cb3989 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.234037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c471eb5-d6f9-49f1-a3ab-3ee652cb3989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.258538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8c471eb5-d6f9-49f1-a3ab-3ee652cb3989 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:01.259045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:01.259855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8c471eb5-d6f9-49f1-a3ab-3ee652cb3989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:01.259972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.260235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.260514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] PATH_INFO: `/v3/users/3921c7a7e4fd41ef9eea262b7a97405a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.261820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] RBAC: Authorizing `identity:get_user(user_id=3921c7a7e4fd41ef9eea262b7a97405a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:01.263542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:01.265315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 20/41] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:00 2022] GET /identity/v3/users/3921c7a7e4fd41ef9eea262b7a97405a => generated 247 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:01.271173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:01.340808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 22/42] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:01.364960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c471eb5-d6f9-49f1-a3ab-3ee652cb3989 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:01.389519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e232-768f-4acc-aab0-036f9f67c45f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:01.397256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:01.431355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:01.432172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.432476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.432737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.433971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:01.435607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:01.437368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 21/43] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:01.442877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:01.449178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:01.483892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c471eb5-d6f9-49f1-a3ab-3ee652cb3989 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:01.518350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:01.519183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.519453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.519736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.521218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:01.522694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:01.524353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 23/44] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:01.529891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:01.543083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3695-3491-43c3-92d5-3f63699c2a1c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:01.577015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d2051-850a-4de3-a391-e6d9650db4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:01.577946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2051-850a-4de3-a391-e6d9650db4fb None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.578239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2051-850a-4de3-a391-e6d9650db4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.578525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2051-850a-4de3-a391-e6d9650db4fb None admin] PATH_INFO: `/v3/domains/default/users/3921c7a7e4fd41ef9eea262b7a97405a/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.580636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d2051-850a-4de3-a391-e6d9650db4fb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3921c7a7e4fd41ef9eea262b7a97405a, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:01.582155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d2051-850a-4de3-a391-e6d9650db4fb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:01.596885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 22/45] 10.222.0.25 () {58 vars in 1361 bytes} [Tue Jul 12 06:59:01 2022] PUT /identity/v3/domains/default/users/3921c7a7e4fd41ef9eea262b7a97405a/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 06:59:01.631211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861911d-956d-447e-b14d-2fb27ee1457e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:01.668084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3d960-cbec-4096-ab08-19e2939c4a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:01.668885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3d960-cbec-4096-ab08-19e2939c4a4b None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:01.669197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3d960-cbec-4096-ab08-19e2939c4a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:01.669495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3d960-cbec-4096-ab08-19e2939c4a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:01.670216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3d960-cbec-4096-ab08-19e2939c4a4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:01.671727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3d960-cbec-4096-ab08-19e2939c4a4b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:01.676855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:01.702200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 06:59:01 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:03.606326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced1b6-d933-4cce-8a71-f1de7c3fc278 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:03.606869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced1b6-d933-4cce-8a71-f1de7c3fc278 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:03.606869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced1b6-d933-4cce-8a71-f1de7c3fc278 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:03.608286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 23/47] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:03.613691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184faef-a8d7-4713-977e-01957edb3a67 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:03.614110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184faef-a8d7-4713-977e-01957edb3a67 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:03.614246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184faef-a8d7-4713-977e-01957edb3a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:03.632303 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a184faef-a8d7-4713-977e-01957edb3a67 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:03.633535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a184faef-a8d7-4713-977e-01957edb3a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:03.701638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305c6f-a536-4f66-a37a-db26b838542e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:03.701917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305c6f-a536-4f66-a37a-db26b838542e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:03.702204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305c6f-a536-4f66-a37a-db26b838542e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:03.703615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 24/48] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:03.704655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 25/49] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:03.708883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6323-2b1c-412c-81e6-451f33fbf3de None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:03.709208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6323-2b1c-412c-81e6-451f33fbf3de None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:03.709495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6323-2b1c-412c-81e6-451f33fbf3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:03.726296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d30f6323-2b1c-412c-81e6-451f33fbf3de None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:03.727335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184faef-a8d7-4713-977e-01957edb3a67 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:03.727493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d30f6323-2b1c-412c-81e6-451f33fbf3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:03.755155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:03.802002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 25/50] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:03.826489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6323-2b1c-412c-81e6-451f33fbf3de None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:03.930697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6323-2b1c-412c-81e6-451f33fbf3de None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:03.936337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184faef-a8d7-4713-977e-01957edb3a67 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:03.953563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:03.964510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:03.965350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:03.965657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:03.965929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:03.967105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:03.968564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:03.970253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 26/51] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:03.975760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:03.975886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:03.976642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:03.976915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:03.977216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:03.989192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:03.998232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:03.999082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:04.000185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 26/52] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:04.005297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:04.009127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:04.076470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336e6c5-1d9b-4a01-8fee-446a97af301f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:04.104331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16238060-346b-4f44-9ab5-035f56087aeb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:04.109189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:04.109856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:04.110139 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:04.110415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:04.111124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:04.112603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:04.138231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 27/53] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Jul 12 06:59:03 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 06:59:04.138795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:04.139577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:04.139858 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:04.140140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:04.140898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:04.141247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:04.142674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:04.156569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 27/54] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:04.162701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:04.256066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:04.274633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d55dc6-a868-4de2-b219-e3fd1f4d0aa1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:04.310448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:04.311347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:04.311634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:04.311929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] PATH_INFO: `/v3/users/3921c7a7e4fd41ef9eea262b7a97405a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:04.313225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] RBAC: Authorizing `identity:get_user(user_id=3921c7a7e4fd41ef9eea262b7a97405a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:04.314717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:04.316607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 28/55] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:04 2022] GET /identity/v3/users/3921c7a7e4fd41ef9eea262b7a97405a => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:04.322200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:04.364495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:04.424197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078754a-021a-410a-88f7-7edd87b4561b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:04.459592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e03f5d-7f30-404c-8085-f9766951a9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:04.460401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e03f5d-7f30-404c-8085-f9766951a9ee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:04.460685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e03f5d-7f30-404c-8085-f9766951a9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:04.460959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e03f5d-7f30-404c-8085-f9766951a9ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:04.462212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e03f5d-7f30-404c-8085-f9766951a9ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:04.463690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e03f5d-7f30-404c-8085-f9766951a9ee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:04.465382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 28/56] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:04.471527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:04.574682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1700-280d-4328-b8d6-71925f5d09a5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:04.610620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461e9c0-17dd-409b-9d49-26049a1b950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:04.611423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e9c0-17dd-409b-9d49-26049a1b950a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:04.611767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e9c0-17dd-409b-9d49-26049a1b950a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:04.612102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e9c0-17dd-409b-9d49-26049a1b950a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:04.612993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461e9c0-17dd-409b-9d49-26049a1b950a None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.domain.id=default, user.id=3921c7a7e4fd41ef9eea262b7a97405a) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:04.613337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461e9c0-17dd-409b-9d49-26049a1b950a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:04.614809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461e9c0-17dd-409b-9d49-26049a1b950a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:04.621236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 29/57] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Jul 12 06:59:04 2022] GET /identity/v3/role_assignments?user.id=3921c7a7e4fd41ef9eea262b7a97405a&scope.domain.id=default&role.id=aaca343d2e4a43c88674834f08499f15 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:06.158371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff128f-de3d-4c60-b718-c8c139376d37 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:06.159198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff128f-de3d-4c60-b718-c8c139376d37 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:06.159198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff128f-de3d-4c60-b718-c8c139376d37 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:06.161519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 29/58] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:06.167691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8f82e-3d3b-4f4b-a54b-d73b9d5de166 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:06.168108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8f82e-3d3b-4f4b-a54b-d73b9d5de166 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:06.168320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8f82e-3d3b-4f4b-a54b-d73b9d5de166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:06.186003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-32c8f82e-3d3b-4f4b-a54b-d73b9d5de166 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:06.187208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-32c8f82e-3d3b-4f4b-a54b-d73b9d5de166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:06.200495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:06.261081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 30/59] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:06.298758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff128f-de3d-4c60-b718-c8c139376d37 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:06.419157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff128f-de3d-4c60-b718-c8c139376d37 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:06.429297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:06.456525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e881e4-5b3f-4a55-a1b6-661ae32d95e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:06.457350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e881e4-5b3f-4a55-a1b6-661ae32d95e9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:06.457630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e881e4-5b3f-4a55-a1b6-661ae32d95e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:06.457898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e881e4-5b3f-4a55-a1b6-661ae32d95e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:06.459108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e881e4-5b3f-4a55-a1b6-661ae32d95e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:06.460567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e881e4-5b3f-4a55-a1b6-661ae32d95e9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:06.462211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 30/60] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:06.468056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8f82e-3d3b-4f4b-a54b-d73b9d5de166 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:06.574738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8f82e-3d3b-4f4b-a54b-d73b9d5de166 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:06.611988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad67a4-e725-44f9-9d02-29cf00fe1b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:06.612829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad67a4-e725-44f9-9d02-29cf00fe1b0f None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:06.613289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad67a4-e725-44f9-9d02-29cf00fe1b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:06.613653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad67a4-e725-44f9-9d02-29cf00fe1b0f None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:06.614807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad67a4-e725-44f9-9d02-29cf00fe1b0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:06.617110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad67a4-e725-44f9-9d02-29cf00fe1b0f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:06.646385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 31/61] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 06:59:06 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:09.866617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fcf8a-70b9-47c5-9ec6-f878236e5f5c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:09.867178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fcf8a-70b9-47c5-9ec6-f878236e5f5c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:09.867178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fcf8a-70b9-47c5-9ec6-f878236e5f5c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:09.868544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 31/62] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:09.874015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc5dfa-15ba-4097-8579-6e960218e99b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:09.875345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc5dfa-15ba-4097-8579-6e960218e99b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:09.875842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc5dfa-15ba-4097-8579-6e960218e99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:09.883984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:09.902101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-70cc5dfa-15ba-4097-8579-6e960218e99b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:09.903333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-70cc5dfa-15ba-4097-8579-6e960218e99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:09.981991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e62a-4c36-4a60-a278-3a1cf02bca31 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:09.982621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e62a-4c36-4a60-a278-3a1cf02bca31 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:09.983542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e62a-4c36-4a60-a278-3a1cf02bca31 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:09.985163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 32/63] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:09.990744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57879a2-cd52-4988-87eb-909a2f891b64 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:09.991212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57879a2-cd52-4988-87eb-909a2f891b64 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:09.991632 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57879a2-cd52-4988-87eb-909a2f891b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.014234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 32/64] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:10.025264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b57879a2-cd52-4988-87eb-909a2f891b64 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:10.026491 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b57879a2-cd52-4988-87eb-909a2f891b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:10.039026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc5dfa-15ba-4097-8579-6e960218e99b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.151357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 33/65] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:10.153874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097eb44-6e73-4fa8-a3b4-d29e54d18ea7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.154563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097eb44-6e73-4fa8-a3b4-d29e54d18ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.155071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097eb44-6e73-4fa8-a3b4-d29e54d18ea7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.156862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 34/66] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:10.162783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82703171-c420-47f5-b935-621f0141aba6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.163260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82703171-c420-47f5-b935-621f0141aba6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.163949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82703171-c420-47f5-b935-621f0141aba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.180023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc5dfa-15ba-4097-8579-6e960218e99b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.186283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-82703171-c420-47f5-b935-621f0141aba6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:10.187884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-82703171-c420-47f5-b935-621f0141aba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:10.221274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:10.230337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.231453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.231934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.232399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.243933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:10.252235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:10.253126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:10.253899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:10.261233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 33/67] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:10.262119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.273800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 35/68] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:10.275453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82703171-c420-47f5-b935-621f0141aba6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.314632 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:10.382322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216c96c-5af6-4f7f-bd13-68ba69103ec8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.390927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82703171-c420-47f5-b935-621f0141aba6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.420688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236b261-4945-4720-a3c8-3acf3989c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.421565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236b261-4945-4720-a3c8-3acf3989c236 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.421853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236b261-4945-4720-a3c8-3acf3989c236 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.422133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236b261-4945-4720-a3c8-3acf3989c236 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.427103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.427780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.428060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.428333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.429092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:10.429400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:10.430783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:10.432864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236b261-4945-4720-a3c8-3acf3989c236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:10.439897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 36/69] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:10.440702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-9236b261-4945-4720-a3c8-3acf3989c236 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:10.442135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:10.443010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:10.443786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:10.447193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 34/70] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:10.449088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa7dd-3648-4e48-9d24-7470fe1dbd9c None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.449578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa7dd-3648-4e48-9d24-7470fe1dbd9c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.450051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa7dd-3648-4e48-9d24-7470fe1dbd9c None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.451541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 35/71] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:10.453850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fa7dd-3648-4e48-9d24-7470fe1dbd9c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.509451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:10.544163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:10.547948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29428aa-7653-4eb0-b9fd-3051f19ba680 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.562335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fa7dd-3648-4e48-9d24-7470fe1dbd9c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.581515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.582335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.582619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.582880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.592943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:10.597698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7879c0a-3c62-420c-aa17-f87bca0aab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.598588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7879c0a-3c62-420c-aa17-f87bca0aab83 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.598877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7879c0a-3c62-420c-aa17-f87bca0aab83 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.599162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7879c0a-3c62-420c-aa17-f87bca0aab83 None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:10.600989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:10.602046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:10.602869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 37/72] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:10.603506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.620576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7879c0a-3c62-420c-aa17-f87bca0aab83 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:10.622126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7879c0a-3c62-420c-aa17-f87bca0aab83 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:10.624132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 36/73] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:10.625662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b58a-3e75-4731-b59f-6f19a9ff84b4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.625940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b58a-3e75-4731-b59f-6f19a9ff84b4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.626235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b58a-3e75-4731-b59f-6f19a9ff84b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.643632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b6a7b58a-3e75-4731-b59f-6f19a9ff84b4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:10.644836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b6a7b58a-3e75-4731-b59f-6f19a9ff84b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:10.709060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f74d2-dd76-4b3b-97ec-605761e9974d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.710664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 37/74] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:10.711967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e850a-eb85-45df-8436-e60496d32694 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.712259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e850a-eb85-45df-8436-e60496d32694 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.712540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e850a-eb85-45df-8436-e60496d32694 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.713856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 38/75] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:10.714554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e850a-eb85-45df-8436-e60496d32694 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.750855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.751538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.751788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.752073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.752831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:10.753196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:10.754572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:10.762705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 38/76] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:10.763484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.769656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:10.784713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:10.830407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e850a-eb85-45df-8436-e60496d32694 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.863628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64b881-143a-4e73-a18b-19b3071a2b58 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:10.865015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff9ad7-77e9-44b5-ab23-bb23e083a7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.865699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff9ad7-77e9-44b5-ab23-bb23e083a7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.865991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff9ad7-77e9-44b5-ab23-bb23e083a7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.866267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff9ad7-77e9-44b5-ab23-bb23e083a7e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.867054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff9ad7-77e9-44b5-ab23-bb23e083a7e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:10.867346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff9ad7-77e9-44b5-ab23-bb23e083a7e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:10.868753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff9ad7-77e9-44b5-ab23-bb23e083a7e4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:10.877009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 39/77] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:10.878179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017794-e9e5-4d26-a5af-c2bb183cb61d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.878467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017794-e9e5-4d26-a5af-c2bb183cb61d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.878750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95017794-e9e5-4d26-a5af-c2bb183cb61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.895116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-95017794-e9e5-4d26-a5af-c2bb183cb61d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:10.896308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-95017794-e9e5-4d26-a5af-c2bb183cb61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:10.898574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:10.899234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:10.899515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:10.899808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:10.962686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 40/78] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:10.963448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95017794-e9e5-4d26-a5af-c2bb183cb61d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:10.973336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:10.975482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:10.999208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.004091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.070366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95017794-e9e5-4d26-a5af-c2bb183cb61d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.080655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 39/79] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:11.081521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.110195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.111008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.111294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.111571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.121869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:11.129706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:11.130538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:11.131305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 41/80] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:11.131935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.181080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612dfc-8c1e-49ff-beb5-305b58e92188 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.200935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.215710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.216532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.216840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.217665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] PATH_INFO: `/v3/users/3921c7a7e4fd41ef9eea262b7a97405a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.219124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] RBAC: Authorizing `identity:get_user(user_id=3921c7a7e4fd41ef9eea262b7a97405a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.220525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.220648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.222482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 40/81] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/users/3921c7a7e4fd41ef9eea262b7a97405a => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:11.223311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.231626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea5d63-577c-44eb-a418-fa023f161f32 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.270294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.271193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.271472 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.271747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.272957 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.274461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.276102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 42/82] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:11.276852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.323933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec082-7ece-4747-9860-0c8f29d323b8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.359107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.359958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.360208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.360487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.361704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.363182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.364822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 41/83] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:11.365609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.382578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be468f-d161-4829-bf05-606a4d655936 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.415204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.416100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.416382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.416655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.417975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.419451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.421248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 43/84] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:11.421781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.422033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.450607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.474206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1e26a-ca5c-4600-b3d8-2e3b0b501a4f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.508913 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.509622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.509908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.510183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.510940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:11.511245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.512638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.520982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 42/85] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:11.521812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.531096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227574d1-e692-4816-b8e0-33f57707e830 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.566316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.567136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.567420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.567691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.633926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6a741-57ae-4611-af02-60b781f5b517 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.650449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.651824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.652717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.670872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.671543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.671845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.672114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.680706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.745131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.746619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.758979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 44/86] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:11.759742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.859499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 43/87] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:11.860358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.873343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ac04-221d-4294-8c24-cbab42220386 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:11.883157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.909306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:11.918927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:11.919604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:11.919885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:11.920158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:11.921038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:11.922711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:11.944290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 45/88] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 06:59:11 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:11.945508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:11.983004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887c5d2-3671-4e1f-b4ff-2289bf21f3d1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.030095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.030670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.030953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.031319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.032255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=19009ab53c574545b66af5787e770785, user.id=4b27ececfd83412c9c497073beaf72f8) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:12.032507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.034056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.053087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 44/89] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/role_assignments?user.id=4b27ececfd83412c9c497073beaf72f8&scope.project.id=19009ab53c574545b66af5787e770785&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:12.053087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:12.098959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9268-97e6-4227-8625-d92ec563609b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.164778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.165757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.166135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.166316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.168014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.170890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.177335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 46/90] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:11 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:12.177335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:12.209599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:12.243809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f1370-3bda-42b8-8d37-87589e0951e4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.261545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:12.298818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.299440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.299730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.300007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.301847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.303337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.305138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 45/91] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:12 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:12.306000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:12.329327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca6985-796a-4510-84d3-41435fbe71fe None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.383921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.384817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.385131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.385414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.386758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.388282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.390862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 47/92] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:12 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:12.391679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:12.432954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d1f1-f888-4bdb-b023-4499895cbb04 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.482469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.483212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.484242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.484819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.517503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:12.538525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b8c16-3f21-47c5-85d5-a0b5c9f2a9a2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.564835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:12.580125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.581564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.584093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.584805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.585123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.585407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.586259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=19009ab53c574545b66af5787e770785, user.id=3921c7a7e4fd41ef9eea262b7a97405a) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:12.586588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.588173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.597165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 48/93] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:12 2022] GET /identity/v3/role_assignments?user.id=3921c7a7e4fd41ef9eea262b7a97405a&scope.project.id=19009ab53c574545b66af5787e770785&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:12.597849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:12.695928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 46/94] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:12.702196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:12.759222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:12.760098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab41ea-2598-4db2-bc94-869b23d5de7d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.815452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fa535-2231-4f84-83fd-e50cb081070c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:12.838766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.839454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.839739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.840017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.840867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] RBAC: Adding query filter params (role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=19009ab53c574545b66af5787e770785, user.id=4b27ececfd83412c9c497073beaf72f8) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:12.841213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.843246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.853589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d11c0c-5de3-4560-a1b5-73f48145f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:12.854478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d11c0c-5de3-4560-a1b5-73f48145f930 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:12.854753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d11c0c-5de3-4560-a1b5-73f48145f930 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:12.855025 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d11c0c-5de3-4560-a1b5-73f48145f930 None admin] PATH_INFO: `/v3/projects/3cd65156186f4a52826fa2bd037a6f33` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:12.856290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d11c0c-5de3-4560-a1b5-73f48145f930 None admin] RBAC: Authorizing `identity:get_project(project_id=3cd65156186f4a52826fa2bd037a6f33)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:12.857788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 49/95] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:12 2022] GET /identity/v3/role_assignments?user.id=4b27ececfd83412c9c497073beaf72f8&scope.project.id=19009ab53c574545b66af5787e770785&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:12.858291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d11c0c-5de3-4560-a1b5-73f48145f930 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:12.865126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 47/96] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:12 2022] GET /identity/v3/projects/3cd65156186f4a52826fa2bd037a6f33 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:12.866597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:12.940374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:13.021526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50051b9-2174-4d46-968d-cd7d5944a48e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:13.057438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f370865-bc11-41fb-90da-a6d577d4eaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:13.058122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f370865-bc11-41fb-90da-a6d577d4eaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:13.058403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f370865-bc11-41fb-90da-a6d577d4eaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:13.058677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f370865-bc11-41fb-90da-a6d577d4eaa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:13.059524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f370865-bc11-41fb-90da-a6d577d4eaa4 None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=3cd65156186f4a52826fa2bd037a6f33, user.id=4b27ececfd83412c9c497073beaf72f8) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:13.059835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f370865-bc11-41fb-90da-a6d577d4eaa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:13.061457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f370865-bc11-41fb-90da-a6d577d4eaa4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:13.071595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 50/97] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:12 2022] GET /identity/v3/role_assignments?user.id=4b27ececfd83412c9c497073beaf72f8&scope.project.id=3cd65156186f4a52826fa2bd037a6f33&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:14.673183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d6889-2499-4246-9bf2-e1772189489d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:14.673709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d6889-2499-4246-9bf2-e1772189489d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:14.673783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d6889-2499-4246-9bf2-e1772189489d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:14.675281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 48/98] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:14.680901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffa265-b385-47a7-a265-f262547f5a4e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:14.681348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffa265-b385-47a7-a265-f262547f5a4e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:14.681510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffa265-b385-47a7-a265-f262547f5a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:14.702297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cdffa265-b385-47a7-a265-f262547f5a4e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:14.703634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cdffa265-b385-47a7-a265-f262547f5a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:14.794409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 51/99] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:14.829356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d6889-2499-4246-9bf2-e1772189489d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:14.940041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d6889-2499-4246-9bf2-e1772189489d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:14.965672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:14.979044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaf48c-b4e0-463c-a458-229c41399f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:14.979912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf48c-b4e0-463c-a458-229c41399f71 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:14.980149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf48c-b4e0-463c-a458-229c41399f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:14.980426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf48c-b4e0-463c-a458-229c41399f71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:14.992455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baaf48c-b4e0-463c-a458-229c41399f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5baaf48c-b4e0-463c-a458-229c41399f71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:15.002002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:15.002904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:15.009696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 49/100] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:15.019283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffa265-b385-47a7-a265-f262547f5a4e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.065607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:15.128716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4549a-0158-4fd8-b9b1-e8e787ac9e6e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.129066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4549a-0158-4fd8-b9b1-e8e787ac9e6e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.129312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4549a-0158-4fd8-b9b1-e8e787ac9e6e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.130817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 50/101] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:15.136156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0dccd-5425-4b5e-920f-3d248a79c4a2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.136442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0dccd-5425-4b5e-920f-3d248a79c4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.136724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0dccd-5425-4b5e-920f-3d248a79c4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.160456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffa265-b385-47a7-a265-f262547f5a4e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:15.162795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1be0dccd-5425-4b5e-920f-3d248a79c4a2 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:15.164011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1be0dccd-5425-4b5e-920f-3d248a79c4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:15.229668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:15.230268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.230554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.230832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.231596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:15.231904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:15.233607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:15.257455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 52/102] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:15.263106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.300224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 51/103] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:15.338046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66027f1b-a7c5-4d7a-a3d6-5cb5afe8f782 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.338670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66027f1b-a7c5-4d7a-a3d6-5cb5afe8f782 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.339172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66027f1b-a7c5-4d7a-a3d6-5cb5afe8f782 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.341194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 52/104] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:15.342226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66027f1b-a7c5-4d7a-a3d6-5cb5afe8f782 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.446297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178c732-5c8c-4486-a8a8-8b48dee946c2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:15.457077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:15.484028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca672cd4-8dd9-4b96-bd7d-354ead22c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:15.484741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672cd4-8dd9-4b96-bd7d-354ead22c9ef None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.485025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672cd4-8dd9-4b96-bd7d-354ead22c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.485324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672cd4-8dd9-4b96-bd7d-354ead22c9ef None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.486581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca672cd4-8dd9-4b96-bd7d-354ead22c9ef None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:15.488073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca672cd4-8dd9-4b96-bd7d-354ead22c9ef None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:15.489767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 53/105] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:15 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:15.491043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d719-4fe5-4105-b8c5-56e50f99d3cd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.491332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d719-4fe5-4105-b8c5-56e50f99d3cd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.491607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d719-4fe5-4105-b8c5-56e50f99d3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.509595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-62c2d719-4fe5-4105-b8c5-56e50f99d3cd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:15.510922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-62c2d719-4fe5-4105-b8c5-56e50f99d3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:15.522688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66027f1b-a7c5-4d7a-a3d6-5cb5afe8f782 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:15.539549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:15.574977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:15.576180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.576684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.577556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.584039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 54/106] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:15.593071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d719-4fe5-4105-b8c5-56e50f99d3cd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.599058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:15.611337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:15.612223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:15.612996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 53/107] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:15.630103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.732909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d719-4fe5-4105-b8c5-56e50f99d3cd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:15.735029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:15.778850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:15.779852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.780136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.780408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.781826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:15.783391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:15.785260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 55/108] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:15 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:15.787215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd481e-5467-4ec5-9755-18acdac0fea9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:15.790623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.823834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c72d61-63b3-45d6-bcb0-ef6627ac5f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:15.824517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72d61-63b3-45d6-bcb0-ef6627ac5f42 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.824803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72d61-63b3-45d6-bcb0-ef6627ac5f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.825102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72d61-63b3-45d6-bcb0-ef6627ac5f42 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.825863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c72d61-63b3-45d6-bcb0-ef6627ac5f42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:15.826169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c72d61-63b3-45d6-bcb0-ef6627ac5f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:15.827641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c72d61-63b3-45d6-bcb0-ef6627ac5f42 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:15.836490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 54/109] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:15.841955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8bdef-6df5-4ac6-953b-2ce234a0f136 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.842250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8bdef-6df5-4ac6-953b-2ce234a0f136 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.842564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8bdef-6df5-4ac6-953b-2ce234a0f136 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.843859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 55/110] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:15.844553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8bdef-6df5-4ac6-953b-2ce234a0f136 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.900535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a70ed-f736-43f4-a5a3-0e9a0df5430e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:15.909505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:15.936391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:15.937250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:15.937531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:15.937802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:15.938999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:15.940467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:15.942502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 56/111] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:15.943279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:15.963162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8bdef-6df5-4ac6-953b-2ce234a0f136 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.003405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:16.012119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6c278-19ec-46fb-8dc0-853709d80ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.013410 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6c278-19ec-46fb-8dc0-853709d80ff2 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.014012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6c278-19ec-46fb-8dc0-853709d80ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.015056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6c278-19ec-46fb-8dc0-853709d80ff2 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785/users/4b27ececfd83412c9c497073beaf72f8/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.017735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6c278-19ec-46fb-8dc0-853709d80ff2 None admin] RBAC: Authorizing `identity:create_grant(project_id=19009ab53c574545b66af5787e770785, user_id=4b27ececfd83412c9c497073beaf72f8, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:16.019718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6c278-19ec-46fb-8dc0-853709d80ff2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:16.039385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 56/112] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:15 2022] PUT /identity/v3/projects/19009ab53c574545b66af5787e770785/users/4b27ececfd83412c9c497073beaf72f8/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:16.041218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b9332-e57e-4c16-be3b-345df041641a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.041700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b9332-e57e-4c16-be3b-345df041641a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.042198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b9332-e57e-4c16-be3b-345df041641a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.072948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d18b9332-e57e-4c16-be3b-345df041641a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:16.074580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d18b9332-e57e-4c16-be3b-345df041641a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:16.106329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4587c64-af01-420d-8751-5f4a407699bb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.156504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365469c1-8c01-4d83-94e1-6e24bc82f436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.157516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365469c1-8c01-4d83-94e1-6e24bc82f436 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.157857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365469c1-8c01-4d83-94e1-6e24bc82f436 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.158146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365469c1-8c01-4d83-94e1-6e24bc82f436 None admin] PATH_INFO: `/v3/users/8cd5535e6fc048368bac4d8812c22c6b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.161468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 57/113] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:16.163504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b9332-e57e-4c16-be3b-345df041641a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:16.174351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365469c1-8c01-4d83-94e1-6e24bc82f436 None admin] RBAC: Authorizing `identity:get_user(user_id=8cd5535e6fc048368bac4d8812c22c6b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:16.175986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365469c1-8c01-4d83-94e1-6e24bc82f436 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:16.178111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 57/114] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:15 2022] GET /identity/v3/users/8cd5535e6fc048368bac4d8812c22c6b => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:16.179565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc06e3-bf40-4f63-8a3b-9e5aab97857e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.179928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc06e3-bf40-4f63-8a3b-9e5aab97857e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.180265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc06e3-bf40-4f63-8a3b-9e5aab97857e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.181910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 58/115] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:16.187058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc06e3-bf40-4f63-8a3b-9e5aab97857e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:16.189442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:16.292436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b9332-e57e-4c16-be3b-345df041641a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.307510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:16.322242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc06e3-bf40-4f63-8a3b-9e5aab97857e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.334220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c5c1b-bb92-4384-93aa-a4d8ec9b834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.334797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c5c1b-bb92-4384-93aa-a4d8ec9b834d None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.335077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c5c1b-bb92-4384-93aa-a4d8ec9b834d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.335346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c5c1b-bb92-4384-93aa-a4d8ec9b834d None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.336057 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c5c1b-bb92-4384-93aa-a4d8ec9b834d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:16.337551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c5c1b-bb92-4384-93aa-a4d8ec9b834d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:16.367181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 58/116] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 06:59:16 2022] POST /identity/v3/projects => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:16.368645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0406c-eb42-4927-8df7-7e8e1ded765e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.368926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0406c-eb42-4927-8df7-7e8e1ded765e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.369232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0406c-eb42-4927-8df7-7e8e1ded765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.370727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c412c59-4160-4491-9124-7b2f6d3b5de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.371791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412c59-4160-4491-9124-7b2f6d3b5de6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.372294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412c59-4160-4491-9124-7b2f6d3b5de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.372874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412c59-4160-4491-9124-7b2f6d3b5de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.390445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-16b0406c-eb42-4927-8df7-7e8e1ded765e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:16.392154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16b0406c-eb42-4927-8df7-7e8e1ded765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:16.471969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 59/117] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:16.472566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c412c59-4160-4491-9124-7b2f6d3b5de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:16.473492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0406c-eb42-4927-8df7-7e8e1ded765e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:16.474321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c412c59-4160-4491-9124-7b2f6d3b5de6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:16.476336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:16.585135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:16.596412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 59/118] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:16.597857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba152-8f22-44ea-a7f7-601c779dc59c None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.598153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba152-8f22-44ea-a7f7-601c779dc59c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.598431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba152-8f22-44ea-a7f7-601c779dc59c None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.599331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0406c-eb42-4927-8df7-7e8e1ded765e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.600002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 60/119] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:16.600730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba152-8f22-44ea-a7f7-601c779dc59c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:16.637704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.638533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.638813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.639086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.652683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:16.661190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:16.662117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:16.662879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 60/120] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:16.663517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:16.742111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:16.748261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba152-8f22-44ea-a7f7-601c779dc59c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.783589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39896469-b73a-4e4f-ae9c-ccebaf438d65 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.795614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.796988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.797463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.798068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.812431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:16.819824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49548c27-53d7-42f0-9ad8-62f9d9df9b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.820820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49548c27-53d7-42f0-9ad8-62f9d9df9b18 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.821225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49548c27-53d7-42f0-9ad8-62f9d9df9b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.821563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49548c27-53d7-42f0-9ad8-62f9d9df9b18 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.822824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49548c27-53d7-42f0-9ad8-62f9d9df9b18 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:16.824313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:16.825161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:16.825947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:16.826609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49548c27-53d7-42f0-9ad8-62f9d9df9b18 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:16.826676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 61/121] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:16 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:16.828609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354be9b-1cbf-438e-9e4f-82c38056bef4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.829001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354be9b-1cbf-438e-9e4f-82c38056bef4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.829375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 61/122] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:16.829472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354be9b-1cbf-438e-9e4f-82c38056bef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:16.830458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:16.848745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d354be9b-1cbf-438e-9e4f-82c38056bef4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:16.850021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d354be9b-1cbf-438e-9e4f-82c38056bef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:16.894971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:16.940416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 62/123] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:16.941984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354be9b-1cbf-438e-9e4f-82c38056bef4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:16.949079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06399c27-c689-4fea-82b2-b0202b38acf3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:16.998185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:16.998971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:16.999260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:16.999588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.000354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:17.000661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.002115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.015757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 62/124] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.016660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.035075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:17.088179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354be9b-1cbf-438e-9e4f-82c38056bef4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.136111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.136946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.137256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.137568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.139031 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] RBAC: Adding query filter params (role.id=4f981d11e2644f2ea3063f3f139a7bb7, scope.project.id=19009ab53c574545b66af5787e770785, user.id=8cd5535e6fc048368bac4d8812c22c6b) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:17.139350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.140680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efde3d-9f52-49e9-8395-af58457ba1cb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.140993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.147955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 63/125] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:16 2022] GET /identity/v3/role_assignments?user.id=8cd5535e6fc048368bac4d8812c22c6b&scope.project.id=19009ab53c574545b66af5787e770785&role.id=4f981d11e2644f2ea3063f3f139a7bb7 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.148756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.163607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:17.191567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.192566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.192861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.193198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.194546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:17.194872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.196614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.209366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 63/126] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.210737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.298726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5160c-d119-4f10-8de2-56bd87935acb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.338341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:17.357451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.359090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.359755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.360846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.383582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:17.400438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:17.401383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:17.402158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 64/127] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:17.402796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.404739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a41d9-a1de-4449-a06e-de3c89762071 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.447659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.449411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.450122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.450593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.452297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.454480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.456643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 64/128] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.457644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.521445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:17.566465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc320f-17c7-4ed5-ae09-85419155ffab None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.604896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.606272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.606753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.607223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.608307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:17.608888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.611249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.621221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 65/129] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.628069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.710845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee6887-d579-4e88-a93a-0f1f740bdf1c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.741830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b92e2-1383-43c4-a4f7-bb77c4eac329 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.769314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:17.785724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.787296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.787891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.788408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.790038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.791715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.793796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 66/130] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.794773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.826336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.827702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.827970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.828291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] PATH_INFO: `/v3/users/3921c7a7e4fd41ef9eea262b7a97405a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.833827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] RBAC: Authorizing `identity:get_user(user_id=3921c7a7e4fd41ef9eea262b7a97405a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.835996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.837321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 65/131] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/users/3921c7a7e4fd41ef9eea262b7a97405a => generated 247 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.838681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:17.906825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461912e-947d-4ebe-ac80-2081f9a7e76e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:17.941862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:17.943112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:17.943589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:17.944106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:17.945571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:17.947344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:17.949677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 67/132] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:17.950712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:18.004215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:18.024982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:18.069573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78081b06-1e89-43d4-ab0e-a51e5b7fe13e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:18.106648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:18.107928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.108648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.109203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.110627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:18.112350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:18.114512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 68/133] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:17 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:18.115471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:18.140324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534b541-802f-4ae4-a250-9351a7d04ffe None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:18.180758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:18.180758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.180758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.180758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785/users/4b27ececfd83412c9c497073beaf72f8/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.183904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] RBAC: Authorizing `identity:create_grant(project_id=19009ab53c574545b66af5787e770785, user_id=4b27ececfd83412c9c497073beaf72f8, role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:18.185498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:18.210662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 66/134] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:17 2022] PUT /identity/v3/projects/19009ab53c574545b66af5787e770785/users/4b27ececfd83412c9c497073beaf72f8/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:18.211491 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:18.310781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9b6d2-8307-4213-a953-77ccf3dce764 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:18.326683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:18.351090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f3953-773b-4a2f-961e-d89d6371b900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:18.351984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3953-773b-4a2f-961e-d89d6371b900 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.352264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3953-773b-4a2f-961e-d89d6371b900 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.352542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3953-773b-4a2f-961e-d89d6371b900 None admin] PATH_INFO: `/v3/projects/3cd65156186f4a52826fa2bd037a6f33` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.353819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3953-773b-4a2f-961e-d89d6371b900 None admin] RBAC: Authorizing `identity:get_project(project_id=3cd65156186f4a52826fa2bd037a6f33)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:18.355359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3953-773b-4a2f-961e-d89d6371b900 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:18.357523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 69/135] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:18 2022] GET /identity/v3/projects/3cd65156186f4a52826fa2bd037a6f33 => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:18.359052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615cad9-5f84-4cdc-9cb7-940312a21157 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.359347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615cad9-5f84-4cdc-9cb7-940312a21157 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.359644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615cad9-5f84-4cdc-9cb7-940312a21157 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.361202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 70/136] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:18.371082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3a9d-878c-4ba7-909c-bb39f1e3a8d0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.371410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3a9d-878c-4ba7-909c-bb39f1e3a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.371645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3a9d-878c-4ba7-909c-bb39f1e3a8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.402154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3caa3a9d-878c-4ba7-909c-bb39f1e3a8d0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:18.403384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3caa3a9d-878c-4ba7-909c-bb39f1e3a8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:18.435901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0559fa-97e9-4f0c-bd4a-39f793615bac None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:18.457033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:18.486817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:18.488090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.488613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.489099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785/users/3921c7a7e4fd41ef9eea262b7a97405a/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.491456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] RBAC: Authorizing `identity:create_grant(project_id=19009ab53c574545b66af5787e770785, user_id=3921c7a7e4fd41ef9eea262b7a97405a, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:18.493268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:18.503496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 71/137] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:18.504821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3a9d-878c-4ba7-909c-bb39f1e3a8d0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:18.517349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 67/138] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:18 2022] PUT /identity/v3/projects/19009ab53c574545b66af5787e770785/users/3921c7a7e4fd41ef9eea262b7a97405a/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:18.532150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:18.709623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3a9d-878c-4ba7-909c-bb39f1e3a8d0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:18.737874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:18.753631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bc516-cdfa-46a8-ba6a-c8f51c85e80f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:18.764271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01ad0e-a528-4d4f-ae89-0bcb66b8bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:18.765737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01ad0e-a528-4d4f-ae89-0bcb66b8bb44 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.766215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01ad0e-a528-4d4f-ae89-0bcb66b8bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.766673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01ad0e-a528-4d4f-ae89-0bcb66b8bb44 None admin] PATH_INFO: `/v3/projects/3cd65156186f4a52826fa2bd037a6f33/users/4b27ececfd83412c9c497073beaf72f8/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.769071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01ad0e-a528-4d4f-ae89-0bcb66b8bb44 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cd65156186f4a52826fa2bd037a6f33, user_id=4b27ececfd83412c9c497073beaf72f8, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:18.771554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01ad0e-a528-4d4f-ae89-0bcb66b8bb44 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:18.806310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 72/139] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:18 2022] PUT /identity/v3/projects/3cd65156186f4a52826fa2bd037a6f33/users/4b27ececfd83412c9c497073beaf72f8/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:18.825431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:18.826318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.826599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.827378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.844401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094493-753a-4634-b778-0f5254cb1773 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.845125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094493-753a-4634-b778-0f5254cb1773 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.845597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094493-753a-4634-b778-0f5254cb1773 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.847477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:18.848202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 73/140] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:18.854511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3e7bb-114f-47bf-8b87-a2cde2b2bfe0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:18.855118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3e7bb-114f-47bf-8b87-a2cde2b2bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:18.855560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:18.856433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:18.857240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 68/141] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:18.858331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3e7bb-114f-47bf-8b87-a2cde2b2bfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:18.862995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:18.892912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-14a3e7bb-114f-47bf-8b87-a2cde2b2bfe0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:18.895919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-14a3e7bb-114f-47bf-8b87-a2cde2b2bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:18.962766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:19.001154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 74/142] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:19.042010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3e7bb-114f-47bf-8b87-a2cde2b2bfe0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:19.127049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee59817-1965-4000-bf23-1e20a391a75b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:19.184097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:19.184797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:19.184902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3e7bb-114f-47bf-8b87-a2cde2b2bfe0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:19.185096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:19.185376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:19.186140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:19.186528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:19.187185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:19.188836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:19.212162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 69/143] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:19.218070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:19.263644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:19.264493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:19.264752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:19.265023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:19.270049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:19.271690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:19.273782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 75/144] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:19.279553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:19.414696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:19.423510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d01b9-99af-4237-a0c6-3b3cbbf5dbbb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:19.492817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:19.493654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:19.494359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:19.494642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:19.496261 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:19.497781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:19.499926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 70/145] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:19 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:19.505770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:19.516069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea60f17-547a-4419-bcb8-b5513437f390 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:19.586249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:19.586935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:19.587210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:19.587478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:19.588197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:19.589686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:19.629548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 76/146] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 06:59:19 2022] POST /identity/v3/users => generated 283 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 06:59:19.679390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6404a4e-4f7e-4289-a69d-63bce14ee708 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:19.731546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23330f-e5a9-4ea4-8af7-559d17a5fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:19.732257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23330f-e5a9-4ea4-8af7-559d17a5fd87 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:19.732540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23330f-e5a9-4ea4-8af7-559d17a5fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:19.732812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23330f-e5a9-4ea4-8af7-559d17a5fd87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:19.794670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:19.828685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23330f-e5a9-4ea4-8af7-559d17a5fd87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:19.830307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23330f-e5a9-4ea4-8af7-559d17a5fd87 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:19.930665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 71/147] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:19.936658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:19.970840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:19.993081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33711437-7ea2-4dea-bfbc-44dd2393573b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:19.993384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33711437-7ea2-4dea-bfbc-44dd2393573b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:19.993662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33711437-7ea2-4dea-bfbc-44dd2393573b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:19.995065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 72/148] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:20.006571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce638-cd49-4b3c-ac8a-c0c6de569c27 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:20.006878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce638-cd49-4b3c-ac8a-c0c6de569c27 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:20.007132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce638-cd49-4b3c-ac8a-c0c6de569c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:20.026747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8d7ce638-cd49-4b3c-ac8a-c0c6de569c27 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:20.027952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8d7ce638-cd49-4b3c-ac8a-c0c6de569c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:20.080648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a13a0-5e65-47af-93d0-c72306746f45 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:20.107464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:20.124031 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 73/149] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:20.139669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:20.140927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:20.141497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:20.141996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:20.143400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:20.145114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:20.147413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 77/150] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:19 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:20.152092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ce638-cd49-4b3c-ac8a-c0c6de569c27 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:20.153764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:20.274388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:20.278035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf6efa-d425-419f-979c-9954b7f5b379 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:20.352451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:20.353038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:20.353341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:20.353614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:20.354460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=19009ab53c574545b66af5787e770785, user.id=4b27ececfd83412c9c497073beaf72f8) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:20.354777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:20.356250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:20.364635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ce638-cd49-4b3c-ac8a-c0c6de569c27 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:20.377264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 78/151] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:20 2022] GET /identity/v3/role_assignments?user.id=4b27ececfd83412c9c497073beaf72f8&scope.project.id=19009ab53c574545b66af5787e770785&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:20.423113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:20.423949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:20.424226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:20.424502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:20.442444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:20.455412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:20.456475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:20.457749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 74/152] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:20.464947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:20.637523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c784c-d3a1-4cd4-af55-815af6306ea1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:20.683626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:20.684570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:20.685004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:20.685454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:20.686367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:20.687464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:20.689030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:20.703756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 79/153] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:20.709315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:20.786150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:20.838605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84b6f6-3703-4526-9128-e4d0c14ba657 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:20.874232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:20.875336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:20.875627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:20.875893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] PATH_INFO: `/v3/users/8cd5535e6fc048368bac4d8812c22c6b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:20.877140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] RBAC: Authorizing `identity:get_user(user_id=8cd5535e6fc048368bac4d8812c22c6b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:20.878610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:20.880446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 75/154] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:20 2022] GET /identity/v3/users/8cd5535e6fc048368bac4d8812c22c6b => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:20.885959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:20.925521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:21.047520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25289-472f-4110-85ae-2407bc3bf2cc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:21.089469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c15ec-1220-4334-a9a5-312b6b56e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:21.090357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c15ec-1220-4334-a9a5-312b6b56e68d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:21.090633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c15ec-1220-4334-a9a5-312b6b56e68d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:21.090911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c15ec-1220-4334-a9a5-312b6b56e68d None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:21.092105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c15ec-1220-4334-a9a5-312b6b56e68d None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:21.093849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c15ec-1220-4334-a9a5-312b6b56e68d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:21.095602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 80/155] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:20 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:21.101469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:21.233132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d0610-f07c-4150-82f8-093ba9b1edd3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:21.265199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:21.282377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313d00d-ed4a-4f6a-8a86-f26c92145aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:21.282994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313d00d-ed4a-4f6a-8a86-f26c92145aef None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:21.283279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313d00d-ed4a-4f6a-8a86-f26c92145aef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:21.283556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313d00d-ed4a-4f6a-8a86-f26c92145aef None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785/users/8cd5535e6fc048368bac4d8812c22c6b/roles/4f981d11e2644f2ea3063f3f139a7bb7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:21.290367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313d00d-ed4a-4f6a-8a86-f26c92145aef None admin] RBAC: Authorizing `identity:create_grant(project_id=19009ab53c574545b66af5787e770785, user_id=8cd5535e6fc048368bac4d8812c22c6b, role_id=4f981d11e2644f2ea3063f3f139a7bb7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:21.291847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313d00d-ed4a-4f6a-8a86-f26c92145aef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:21.320060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 76/156] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:21 2022] PUT /identity/v3/projects/19009ab53c574545b66af5787e770785/users/8cd5535e6fc048368bac4d8812c22c6b/roles/4f981d11e2644f2ea3063f3f139a7bb7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:21.479037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e202-945c-466e-8d31-84e402a4cfce None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:21.479765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e202-945c-466e-8d31-84e402a4cfce None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:21.480251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e202-945c-466e-8d31-84e402a4cfce None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:21.481946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 81/157] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:21.488292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f9c4-4acd-4c9b-9b5b-1823756f62d0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:21.488887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f9c4-4acd-4c9b-9b5b-1823756f62d0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:21.489402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f9c4-4acd-4c9b-9b5b-1823756f62d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:21.507980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3875f9c4-4acd-4c9b-9b5b-1823756f62d0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:21.509597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3875f9c4-4acd-4c9b-9b5b-1823756f62d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:21.601545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 77/158] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:21.640680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6e202-945c-466e-8d31-84e402a4cfce None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:21.741714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:21.760994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6e202-945c-466e-8d31-84e402a4cfce None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:21.785240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136632-1b9f-435f-9eae-9404017a8f09 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:21.785900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136632-1b9f-435f-9eae-9404017a8f09 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:21.786383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136632-1b9f-435f-9eae-9404017a8f09 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:21.788050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 78/159] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:21.798962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562aef5-97f5-4d4d-b00f-35b23c4404b5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:21.799638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562aef5-97f5-4d4d-b00f-35b23c4404b5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:21.800125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562aef5-97f5-4d4d-b00f-35b23c4404b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:21.806737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:21.808451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:21.808451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:21.808451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:21.819214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:21.829085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:21.829984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:21.830813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 82/160] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:21.837827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:21.846073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6562aef5-97f5-4d4d-b00f-35b23c4404b5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:21.847908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6562aef5-97f5-4d4d-b00f-35b23c4404b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:22.013918 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 79/161] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:22.023087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2beac3-b8aa-4bb9-8e6b-789f6202c035 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:22.055543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562aef5-97f5-4d4d-b00f-35b23c4404b5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:22.072722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:22.074623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.075152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.075574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.077674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:22.078321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:22.081151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:22.098713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 83/162] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:22.104422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:22.134226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:22.135203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:22.231708 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562aef5-97f5-4d4d-b00f-35b23c4404b5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:22.282299 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8b9bd-d1fd-4f4d-8f45-79eb2d1464ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:22.283385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b9bd-d1fd-4f4d-8f45-79eb2d1464ad None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.283821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b9bd-d1fd-4f4d-8f45-79eb2d1464ad None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.284232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b9bd-d1fd-4f4d-8f45-79eb2d1464ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.315938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840f919-77b2-44da-83bb-3a1127a9e79b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:22.316206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8b9bd-d1fd-4f4d-8f45-79eb2d1464ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a6d8b9bd-d1fd-4f4d-8f45-79eb2d1464ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:22.324788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:22.325943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:22.326933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:22.334505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 80/163] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:22.335843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1a913-a86d-4ab5-bffb-93ae9904fe2a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.336135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1a913-a86d-4ab5-bffb-93ae9904fe2a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.336413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1a913-a86d-4ab5-bffb-93ae9904fe2a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.337876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 81/164] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:22.343500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef62c1-e1ff-4b8b-908b-7a91ed4b13cd None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.343971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef62c1-e1ff-4b8b-908b-7a91ed4b13cd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.344256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef62c1-e1ff-4b8b-908b-7a91ed4b13cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.365751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:22.366970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.367432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.368115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] PATH_INFO: `/v3/users/3921c7a7e4fd41ef9eea262b7a97405a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.369807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] RBAC: Authorizing `identity:get_user(user_id=3921c7a7e4fd41ef9eea262b7a97405a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:22.372467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:22.377977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 84/165] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:22 2022] GET /identity/v3/users/3921c7a7e4fd41ef9eea262b7a97405a => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:22.380085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-23ef62c1-e1ff-4b8b-908b-7a91ed4b13cd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:22.380572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:22.382298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-23ef62c1-e1ff-4b8b-908b-7a91ed4b13cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:22.503297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 82/166] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:22.504647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef62c1-e1ff-4b8b-908b-7a91ed4b13cd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:22.527568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:22.563799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:22.567654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665530a-a3ac-4fc4-9237-6f71abe9ef9f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:22.634494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:22.635503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.635968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.636426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.638556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:22.639050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:22.641738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:22.663400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 85/167] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:22.664557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:22.712703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef62c1-e1ff-4b8b-908b-7a91ed4b13cd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:22.772475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:22.773179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.773465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.773741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.849681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:22.851121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:22.868357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c6996-016f-4630-93cc-ef5da034adf1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:22.883581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:22.964249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:22.964249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:22.964249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:22.964249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:22.964789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 83/168] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:22.965862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:22.998459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:23.009620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:23.017573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:23.018667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 86/169] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:23.019721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:23.087920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09ecf-a36e-4547-be33-5b67b31e0c16 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:23.120992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:23.127942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:23.129039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.129606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.130225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.131576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:23.133222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:23.135375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 84/170] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:22 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:23.136422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:23.271861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250c11a-ac13-4072-a371-c84048a2991d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:23.275889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c12ad0-09da-4fab-834c-a675c6077824 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:23.320083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:23.320970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.321274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.321549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.322923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:23.324729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:23.326836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 87/171] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:23 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:23.327801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:23.345963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:23.346655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.346936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.347207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.347967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:23.348272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:23.349705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:23.378945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 85/172] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:23.379673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:23.469635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:23.532911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:23.540725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8116583-74a2-4d27-8153-6dbce8a1983c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:23.602314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b788ce-cd87-432f-aed9-9b1a366bed31 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:23.606649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65b796-6b11-4c3f-b8b6-97e28d629b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:23.607317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b796-6b11-4c3f-b8b6-97e28d629b66 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.607600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b796-6b11-4c3f-b8b6-97e28d629b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.607873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b796-6b11-4c3f-b8b6-97e28d629b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.670568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:23.671208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.671488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.671842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.672711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=19009ab53c574545b66af5787e770785, user.id=3921c7a7e4fd41ef9eea262b7a97405a) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:23.673022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:23.674492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:23.683648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 86/173] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:23 2022] GET /identity/v3/role_assignments?user.id=3921c7a7e4fd41ef9eea262b7a97405a&scope.project.id=19009ab53c574545b66af5787e770785&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:23.684249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:23.739391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65b796-6b11-4c3f-b8b6-97e28d629b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:23.741018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65b796-6b11-4c3f-b8b6-97e28d629b66 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:23.836089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a181f1-79fb-424e-b525-413bb365c31e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:23.845074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:23.884542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 88/174] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:23.889473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:23.890292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296546-ccf1-4ff8-bba6-d7568c5d391d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.890587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296546-ccf1-4ff8-bba6-d7568c5d391d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.890867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296546-ccf1-4ff8-bba6-d7568c5d391d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.892520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 89/175] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:23.893967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296546-ccf1-4ff8-bba6-d7568c5d391d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:23.926800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730ce4f-bcf7-490b-bd97-cc9edacc62dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:23.927624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ce4f-bcf7-490b-bd97-cc9edacc62dc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.927899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ce4f-bcf7-490b-bd97-cc9edacc62dc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.928170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ce4f-bcf7-490b-bd97-cc9edacc62dc None admin] PATH_INFO: `/v3/users/4b27ececfd83412c9c497073beaf72f8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.934152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730ce4f-bcf7-490b-bd97-cc9edacc62dc None admin] RBAC: Authorizing `identity:get_user(user_id=4b27ececfd83412c9c497073beaf72f8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:23.935644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730ce4f-bcf7-490b-bd97-cc9edacc62dc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:23.939059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 87/176] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:23 2022] GET /identity/v3/users/4b27ececfd83412c9c497073beaf72f8 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:23.939059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227ac63-43bb-4cea-a23c-678ae19a97f6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:23.939277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227ac63-43bb-4cea-a23c-678ae19a97f6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:23.939566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227ac63-43bb-4cea-a23c-678ae19a97f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:23.966967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8227ac63-43bb-4cea-a23c-678ae19a97f6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:23.968314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8227ac63-43bb-4cea-a23c-678ae19a97f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:24.066928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296546-ccf1-4ff8-bba6-d7568c5d391d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:24.070496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 88/177] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:24.071357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227ac63-43bb-4cea-a23c-678ae19a97f6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:24.132293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:24.133934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.133934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.133934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.137001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:24.138496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:24.141301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 90/178] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:23 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:24.142159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:24.198825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:24.230550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227ac63-43bb-4cea-a23c-678ae19a97f6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:24.261801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:24.277991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:24.279111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.279544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.280160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.283271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024804f7-1764-444c-8214-870857aa5db2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:24.325923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7c885-fe60-402c-a270-33911c215325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:24.327609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7c885-fe60-402c-a270-33911c215325 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7c885-fe60-402c-a270-33911c215325 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.328598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7c885-fe60-402c-a270-33911c215325 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.340494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7c885-fe60-402c-a270-33911c215325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6cf7c885-fe60-402c-a270-33911c215325 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:24.348801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:24.349965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:24.350809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:24.351405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 91/179] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:24.352935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95eb4c-e0dd-46ea-9d2a-35c776e3be4e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.353342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95eb4c-e0dd-46ea-9d2a-35c776e3be4e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.353706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95eb4c-e0dd-46ea-9d2a-35c776e3be4e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.355073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 92/180] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:24.359920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95eb4c-e0dd-46ea-9d2a-35c776e3be4e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:24.372988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:24.374832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:24.478092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:24.487160 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95eb4c-e0dd-46ea-9d2a-35c776e3be4e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:24.505550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 89/181] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:24.511307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:24.534101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21e370-a5f4-464c-9972-edd32d65fe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:24.534764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21e370-a5f4-464c-9972-edd32d65fe7c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.535171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21e370-a5f4-464c-9972-edd32d65fe7c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.535447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21e370-a5f4-464c-9972-edd32d65fe7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.536286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21e370-a5f4-464c-9972-edd32d65fe7c None admin] RBAC: Adding query filter params (role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=19009ab53c574545b66af5787e770785, user.id=4b27ececfd83412c9c497073beaf72f8) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:24.536592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21e370-a5f4-464c-9972-edd32d65fe7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:24.538751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21e370-a5f4-464c-9972-edd32d65fe7c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:24.540584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:24.548240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 93/182] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/role_assignments?user.id=4b27ececfd83412c9c497073beaf72f8&scope.project.id=19009ab53c574545b66af5787e770785&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:24.549553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95d9d5-5cd6-478a-94ab-fab0a2f61825 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.549982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95d9d5-5cd6-478a-94ab-fab0a2f61825 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.550260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95d9d5-5cd6-478a-94ab-fab0a2f61825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.570784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7b95d9d5-5cd6-478a-94ab-fab0a2f61825 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:24.572013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7b95d9d5-5cd6-478a-94ab-fab0a2f61825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:24.639216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6974d-5017-4a1f-a86b-2914b6d57f61 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:24.659414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 94/183] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:24.665188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95d9d5-5cd6-478a-94ab-fab0a2f61825 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:24.689144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1c180-2fb0-4b61-88bd-d15710a624b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:24.689843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c180-2fb0-4b61-88bd-d15710a624b8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.690209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c180-2fb0-4b61-88bd-d15710a624b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.690819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c180-2fb0-4b61-88bd-d15710a624b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.691607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c180-2fb0-4b61-88bd-d15710a624b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:24.692095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c180-2fb0-4b61-88bd-d15710a624b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:24.693771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c180-2fb0-4b61-88bd-d15710a624b8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:24.705655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 90/184] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:24.705655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d8744-3a1d-43a2-ab30-8d70808c3864 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.705655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d8744-3a1d-43a2-ab30-8d70808c3864 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.705655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d8744-3a1d-43a2-ab30-8d70808c3864 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.707293 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 91/185] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:24.708481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30294-b4a7-40e2-8f03-93a0bd4709cf None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.708762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30294-b4a7-40e2-8f03-93a0bd4709cf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.709034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b30294-b4a7-40e2-8f03-93a0bd4709cf None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.710463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 92/186] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:24.711404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b30294-b4a7-40e2-8f03-93a0bd4709cf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:24.763922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:24.774837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95d9d5-5cd6-478a-94ab-fab0a2f61825 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:24.811314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:24.814416 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce663f05-0638-4277-9551-e1eb6ab1f44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:24.815546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce663f05-0638-4277-9551-e1eb6ab1f44b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.815776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce663f05-0638-4277-9551-e1eb6ab1f44b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.816049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce663f05-0638-4277-9551-e1eb6ab1f44b None admin] PATH_INFO: `/v3/projects/3cd65156186f4a52826fa2bd037a6f33` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.817359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce663f05-0638-4277-9551-e1eb6ab1f44b None admin] RBAC: Authorizing `identity:get_project(project_id=3cd65156186f4a52826fa2bd037a6f33)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:24.818818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce663f05-0638-4277-9551-e1eb6ab1f44b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:24.820531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 95/187] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/projects/3cd65156186f4a52826fa2bd037a6f33 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:24.821688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6464ab-dada-4ffa-8c30-4380b9df45f0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.821964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6464ab-dada-4ffa-8c30-4380b9df45f0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.822236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6464ab-dada-4ffa-8c30-4380b9df45f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.834715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b30294-b4a7-40e2-8f03-93a0bd4709cf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:24.839175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0a6464ab-dada-4ffa-8c30-4380b9df45f0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:24.840348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0a6464ab-dada-4ffa-8c30-4380b9df45f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:24.875060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf36b3d-516b-4059-b1bd-1045121d6fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:24.875873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf36b3d-516b-4059-b1bd-1045121d6fdd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.876191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf36b3d-516b-4059-b1bd-1045121d6fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.876429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf36b3d-516b-4059-b1bd-1045121d6fdd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.877972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf36b3d-516b-4059-b1bd-1045121d6fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:24.879596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf36b3d-516b-4059-b1bd-1045121d6fdd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:24.881552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 93/188] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:24.882778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cea9d-4a71-40e9-92db-e5042b49713f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:24.883071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cea9d-4a71-40e9-92db-e5042b49713f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:24.883349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cea9d-4a71-40e9-92db-e5042b49713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:24.900580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c97cea9d-4a71-40e9-92db-e5042b49713f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:24.901822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c97cea9d-4a71-40e9-92db-e5042b49713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:24.916068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 96/189] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:24.916277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6464ab-dada-4ffa-8c30-4380b9df45f0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:24.977527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 94/190] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:24.978173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cea9d-4a71-40e9-92db-e5042b49713f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.021167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.033611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6464ab-dada-4ffa-8c30-4380b9df45f0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.065533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.066685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.067619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.067906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.068159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] PATH_INFO: `/v3/users/386e72b133b242198025d86aeecaad0c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.082173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] RBAC: Authorizing `identity:get_user(user_id=386e72b133b242198025d86aeecaad0c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:25.083678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:25.085154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cea9d-4a71-40e9-92db-e5042b49713f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.085426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 97/191] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/users/386e72b133b242198025d86aeecaad0c => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:25.086171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.119447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.120112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.120394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.120659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.121520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=3cd65156186f4a52826fa2bd037a6f33, user.id=4b27ececfd83412c9c497073beaf72f8) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:25.121824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:25.123255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:25.128954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 95/192] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:24 2022] GET /identity/v3/role_assignments?user.id=4b27ececfd83412c9c497073beaf72f8&scope.project.id=3cd65156186f4a52826fa2bd037a6f33&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:25.129748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.201647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd840e8-86ad-439d-8a55-ff2ad78a75d8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.238050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.238745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.239032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.239309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.240034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:25.241555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:25.261124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e21013-441b-4271-8890-78ba6b6c8959 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.265294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 98/193] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 06:59:25 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:25.265294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.277431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.320189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.322124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.322507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.322833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.343989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:25.352537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:25.353665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:25.354471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 96/194] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:25.355119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.367162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.422578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea103dd6-793c-40e2-b3bc-6e9e46a308ce None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.465291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.466450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.466740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.467163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.479362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:25.487575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:25.488461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 99/195] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:25.489683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.492911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69381a7d-7ccf-498c-97d0-2cf5deb2a8b7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.537436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.538014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.538292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.538565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.570726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.621331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14cb71-ffc7-4493-99b2-75c21d652f35 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.622764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:25.624719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:25.634071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.662177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.662862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.663141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.663419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.664177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:25.664486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:25.665932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:25.675225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 100/196] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:25.676049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.760808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 97/197] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:25.762579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.799707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e5ebb-6671-4e24-bb77-915272ffa739 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.828659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.841081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.841985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.842402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.842899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.843854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:25.844332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:25.845893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:25.854879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 101/198] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:25.855884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.872745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695dfda-2e25-4f44-bdfe-c304440261d6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:25.880915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:25.909942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:25.910839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:25.911120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:25.911391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] PATH_INFO: `/v3/users/8cd5535e6fc048368bac4d8812c22c6b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:25.912598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] RBAC: Authorizing `identity:get_user(user_id=8cd5535e6fc048368bac4d8812c22c6b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:25.914085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:25.916107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 98/199] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/users/8cd5535e6fc048368bac4d8812c22c6b => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:25.917003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:25.963528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac829fe-9429-488f-8074-9b2fa9d939aa None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.004644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.006315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.006648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.006931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.008384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.010620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.017146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 102/200] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:26.018046 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:26.032378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0e24-a06a-4b22-a378-d45da975a23d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.081074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.082140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.082418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.082688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] PATH_INFO: `/v3/users/386e72b133b242198025d86aeecaad0c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.084487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] RBAC: Authorizing `identity:get_user(user_id=386e72b133b242198025d86aeecaad0c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.085964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.088251 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 99/201] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:25 2022] GET /identity/v3/users/386e72b133b242198025d86aeecaad0c => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:26.089148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:26.090171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:26.146784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:26.151311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06957e81-b8c3-4bb7-9baf-26ca9ac834c1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.186188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.186932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.187214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.187485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.213961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff5da-1999-4b51-8347-be2b06ee5522 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.256116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.257292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.257825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.258316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.259361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=386e72b133b242198025d86aeecaad0c) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:26.259955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.261728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.267717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.269236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.277159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 100/202] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:26 2022] GET /identity/v3/role_assignments?user.id=386e72b133b242198025d86aeecaad0c&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:26.278546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:26.333557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:26.389643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 103/203] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:26.395063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:26.473837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a2a75-7852-49df-92eb-da83dcaf5e1b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.482148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:26.508981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf449f-75ef-4e7e-b494-7346e08c5468 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.514272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.515296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.515828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.516338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.551597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.552482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.552768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.553073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.554360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.555911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.558082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 104/204] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:26 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:26.563796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:26.598625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:26.601593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.603222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.693522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7b3b-5c2d-4f97-98fd-303228d0c3ef None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.739566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.740240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.740522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.740803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.741687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] RBAC: Adding query filter params (role.id=4f981d11e2644f2ea3063f3f139a7bb7, scope.project.id=19009ab53c574545b66af5787e770785, user.id=8cd5535e6fc048368bac4d8812c22c6b) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:26.742007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.744005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.751554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 105/205] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:26 2022] GET /identity/v3/role_assignments?user.id=8cd5535e6fc048368bac4d8812c22c6b&scope.project.id=19009ab53c574545b66af5787e770785&role.id=4f981d11e2644f2ea3063f3f139a7bb7 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:26.803702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 101/206] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:26.809651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:26.915271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:26.931550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6825c-301e-415e-bda9-b937f576617b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:26.972052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b83ddf-9579-4620-838e-67e0bb20e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:26.972949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b83ddf-9579-4620-838e-67e0bb20e060 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:26.973256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b83ddf-9579-4620-838e-67e0bb20e060 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:26.973535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b83ddf-9579-4620-838e-67e0bb20e060 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:26.974783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b83ddf-9579-4620-838e-67e0bb20e060 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:26.976272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b83ddf-9579-4620-838e-67e0bb20e060 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:26.978035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 106/207] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:26 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:26.986234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:27.005079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:27.104338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff4ec-231f-413e-bedb-e415fd09a98a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:27.143122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c0a7d-54b9-4873-bdcd-97a3347c2b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:27.143797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c0a7d-54b9-4873-bdcd-97a3347c2b71 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:27.144070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c0a7d-54b9-4873-bdcd-97a3347c2b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:27.144340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c0a7d-54b9-4873-bdcd-97a3347c2b71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:27.145209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c0a7d-54b9-4873-bdcd-97a3347c2b71 None admin] RBAC: Adding query filter params (role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=386e72b133b242198025d86aeecaad0c) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:27.145519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c0a7d-54b9-4873-bdcd-97a3347c2b71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:27.146966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c0a7d-54b9-4873-bdcd-97a3347c2b71 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:27.153108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 102/208] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:26 2022] GET /identity/v3/role_assignments?user.id=386e72b133b242198025d86aeecaad0c&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:27.628378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027d85e-f7d8-47cb-b611-8279f03263e7 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:27.628977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027d85e-f7d8-47cb-b611-8279f03263e7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:27.628977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027d85e-f7d8-47cb-b611-8279f03263e7 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:27.630734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 107/209] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:27.636824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a1f98-4e27-49d2-b714-a7e72982469c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:27.637174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a1f98-4e27-49d2-b714-a7e72982469c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:27.637421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a1f98-4e27-49d2-b714-a7e72982469c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:27.655678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0c9a1f98-4e27-49d2-b714-a7e72982469c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:27.656873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0c9a1f98-4e27-49d2-b714-a7e72982469c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:27.721180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:27.726002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 103/210] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:27.733019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865960d3-74c3-4b55-8fe2-187493211c39 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:27.733722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865960d3-74c3-4b55-8fe2-187493211c39 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:27.734232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865960d3-74c3-4b55-8fe2-187493211c39 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:27.735978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 108/211] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:27.741460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983108-a5d2-485c-8177-e8e228ce9294 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:27.741973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983108-a5d2-485c-8177-e8e228ce9294 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:27.742448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983108-a5d2-485c-8177-e8e228ce9294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:27.753194 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865960d3-74c3-4b55-8fe2-187493211c39 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:27.766536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-02983108-a5d2-485c-8177-e8e228ce9294 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:27.768014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-02983108-a5d2-485c-8177-e8e228ce9294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:27.854230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 104/212] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:27.873949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865960d3-74c3-4b55-8fe2-187493211c39 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:27.878868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983108-a5d2-485c-8177-e8e228ce9294 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:27.918614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:27.919436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:27.919722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:27.920000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:27.931778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:27.941973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:27.942921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:27.943715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 109/213] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:27.950042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:27.977466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:28.005273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983108-a5d2-485c-8177-e8e228ce9294 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:28.041494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:28.049534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:28.050500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.050780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.051076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.052485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:28.054017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:28.056763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 105/214] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:28.063232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:28.072754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d46c40-5661-4440-8d89-e4afefeeaf48 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:28.113836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:28.114520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.114806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.115079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.115837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:28.116146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:28.117598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:28.131786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 110/215] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:28.132602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:28.190841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa3a97-5a31-449a-a457-8b32f0a88ff1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:28.228577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:28.229301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.229584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.229856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.230571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:28.232021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:28.235795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:28.265714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160df65-370a-4611-a32b-03cf73962235 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:28.270828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 106/216] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 06:59:28 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:28.314338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6c8c2-744e-4438-939b-01e95cb11c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:28.315814 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6c8c2-744e-4438-939b-01e95cb11c93 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.316332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6c8c2-744e-4438-939b-01e95cb11c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.316808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6c8c2-744e-4438-939b-01e95cb11c93 None admin] PATH_INFO: `/v3/users/0d2613c2bb3848df8cd4f9f2104da2cd` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.336020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6c8c2-744e-4438-939b-01e95cb11c93 None admin] RBAC: Authorizing `identity:get_user(user_id=0d2613c2bb3848df8cd4f9f2104da2cd)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:28.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6c8c2-744e-4438-939b-01e95cb11c93 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:28.340231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 111/217] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:28 2022] GET /identity/v3/users/0d2613c2bb3848df8cd4f9f2104da2cd => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:28.346604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:28.412374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:28.462634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087efc8-6935-43d8-be27-45661760a929 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:28.498752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:28.499534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.499534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.500112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.570051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:28.571531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:28.580217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043a7f-f0fd-4d5a-aa1b-947b43e4a4b0 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.580500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043a7f-f0fd-4d5a-aa1b-947b43e4a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.580777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043a7f-f0fd-4d5a-aa1b-947b43e4a4b0 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.582224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 112/218] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:28.587376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9dc69-c1e9-466f-a71f-c981d11922f0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.587664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9dc69-c1e9-466f-a71f-c981d11922f0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.587942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9dc69-c1e9-466f-a71f-c981d11922f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.607482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3aa9dc69-c1e9-466f-a71f-c981d11922f0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:28.608711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3aa9dc69-c1e9-466f-a71f-c981d11922f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:28.638334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:28.701942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 107/219] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:28.705035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 113/220] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:28.707655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:28.730833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9dc69-c1e9-466f-a71f-c981d11922f0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:28.811331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:28.825982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0a99-203c-48a1-a6c1-0a44615f55c1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:28.873919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9dc69-c1e9-466f-a71f-c981d11922f0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:28.887094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:28.888321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.888823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.889332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.892837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:28.894635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:28.899053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 108/221] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:28 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:28.904768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:28.943007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:28.944453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:28.944948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:28.945444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:28.947776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:28.967314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:28.984702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:28.985701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:28.986518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 114/222] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:28.994340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:29.091686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1c17e-46bc-42ea-8967-250a8171305d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:29.158432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:29.159261 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.159636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.160008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.160937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=0d2613c2bb3848df8cd4f9f2104da2cd) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:29.161416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:29.163773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:29.173031 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 109/223] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:28 2022] GET /identity/v3/role_assignments?user.id=0d2613c2bb3848df8cd4f9f2104da2cd&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:29.174881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8714-112b-42e1-b99a-ebe0cd3790d3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:29.201373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:29.214404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f46bc-8b4d-46df-9ce5-96dde9d19741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:29.215087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f46bc-8b4d-46df-9ce5-96dde9d19741 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.215373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f46bc-8b4d-46df-9ce5-96dde9d19741 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.215649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f46bc-8b4d-46df-9ce5-96dde9d19741 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.216413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f46bc-8b4d-46df-9ce5-96dde9d19741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:29.216718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f46bc-8b4d-46df-9ce5-96dde9d19741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:29.218346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f46bc-8b4d-46df-9ce5-96dde9d19741 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:29.228338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 115/224] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:29.233990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:29.321831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdade1-2a0b-43fb-9c90-983453e4b4dd None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.322152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdade1-2a0b-43fb-9c90-983453e4b4dd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.322397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdade1-2a0b-43fb-9c90-983453e4b4dd None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.324509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 116/225] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:29.331601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c2a48-ae12-4ebf-b9cc-2a18799b13dd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.332264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c2a48-ae12-4ebf-b9cc-2a18799b13dd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.332933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c2a48-ae12-4ebf-b9cc-2a18799b13dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.362054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:29.370145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-993c2a48-ae12-4ebf-b9cc-2a18799b13dd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:29.371367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-993c2a48-ae12-4ebf-b9cc-2a18799b13dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:29.376135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615214fa-6061-4936-87ca-032dc0a74b71 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:29.432008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:29.433624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.434979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.435453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] PATH_INFO: `/v3/users/386e72b133b242198025d86aeecaad0c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.436936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] RBAC: Authorizing `identity:get_user(user_id=386e72b133b242198025d86aeecaad0c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:29.438841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:29.441335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 110/226] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:29 2022] GET /identity/v3/users/386e72b133b242198025d86aeecaad0c => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:29.447802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:29.515574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 117/227] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:29.551608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c2a48-ae12-4ebf-b9cc-2a18799b13dd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:29.560077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704062c7-1cce-45b0-bfaf-a2f664b0ff28 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:29.597524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:29.598727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.599223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.599752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.601297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:29.603271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:29.605219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 111/228] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:29 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:29.611341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:29.657419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:29.760461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8f67c-5d6f-423c-a52f-b7eab5b2c0bf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:29.764090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c2a48-ae12-4ebf-b9cc-2a18799b13dd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:29.796986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:29.827071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:29.828265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.828552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/386e72b133b242198025d86aeecaad0c/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.830917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03bdf0b01b44e5abf418124cac9cab, user_id=386e72b133b242198025d86aeecaad0c, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:29.831368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:29.832433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:29.832546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:29.833001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:29.833487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:29.862833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 112/229] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:29 2022] PUT /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/386e72b133b242198025d86aeecaad0c/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 06:59:29.864259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:29.881328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:29.882282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:29.883084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 118/230] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:29.889156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:30.067467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:30.091850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5460b-1676-4360-81e5-54a4ed44de25 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:30.128306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:30.128988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:30.129304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:30.129580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:30.130324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:30.130635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:30.132045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:30.141070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 113/231] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:30.146641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:30.252854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1963ce-f03a-4ef0-8b44-6037445c0755 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:30.288389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:30.289309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:30.289593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:30.289867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] PATH_INFO: `/v3/users/386e72b133b242198025d86aeecaad0c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:30.291128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] RBAC: Authorizing `identity:get_user(user_id=386e72b133b242198025d86aeecaad0c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:30.292632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:30.294613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 119/232] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:30 2022] GET /identity/v3/users/386e72b133b242198025d86aeecaad0c => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:30.300395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:30.408511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92640b4f-6785-4baa-ba2c-4ef5c489f781 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:30.443798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaba1e-9b52-4ff9-9e62-1da3e645b93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:30.444685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaba1e-9b52-4ff9-9e62-1da3e645b93b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:30.444963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaba1e-9b52-4ff9-9e62-1da3e645b93b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:30.445269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaba1e-9b52-4ff9-9e62-1da3e645b93b None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:30.446457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbaba1e-9b52-4ff9-9e62-1da3e645b93b None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:30.447938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbaba1e-9b52-4ff9-9e62-1da3e645b93b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:30.450060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 114/233] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:30 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:30.456158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:30.505688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:30.568518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b8568-10f4-45c5-a3bb-6fcb2f6373f3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:30.604334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144d4ff-5c9b-4b7b-aa46-a97376ac5b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:30.605289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144d4ff-5c9b-4b7b-aa46-a97376ac5b8e None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:30.605574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144d4ff-5c9b-4b7b-aa46-a97376ac5b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:30.605852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144d4ff-5c9b-4b7b-aa46-a97376ac5b8e None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/386e72b133b242198025d86aeecaad0c/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:30.608093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144d4ff-5c9b-4b7b-aa46-a97376ac5b8e None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03bdf0b01b44e5abf418124cac9cab, user_id=386e72b133b242198025d86aeecaad0c, role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:30.609808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144d4ff-5c9b-4b7b-aa46-a97376ac5b8e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:30.632796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 120/234] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:30 2022] PUT /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/386e72b133b242198025d86aeecaad0c/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:30.954652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf505fe-f660-4213-9892-5103f9964b8d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:30.955056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf505fe-f660-4213-9892-5103f9964b8d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:30.955221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf505fe-f660-4213-9892-5103f9964b8d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:30.956586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 115/235] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:30.962392 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900baeda-708f-45f6-98bd-4590dedddb5f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:30.962980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900baeda-708f-45f6-98bd-4590dedddb5f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:30.963129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900baeda-708f-45f6-98bd-4590dedddb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:30.980660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-900baeda-708f-45f6-98bd-4590dedddb5f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:30.986066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-900baeda-708f-45f6-98bd-4590dedddb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:31.079820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 121/236] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:31.116288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf505fe-f660-4213-9892-5103f9964b8d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:31.133119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:31.208616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6547-24ba-4bf1-af56-de5c41e3af5a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.208919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6547-24ba-4bf1-af56-de5c41e3af5a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.209105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6547-24ba-4bf1-af56-de5c41e3af5a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.210524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 122/237] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:31.215626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35162f7-164c-4ebb-b079-5898a50974f2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.215913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35162f7-164c-4ebb-b079-5898a50974f2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.216189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35162f7-164c-4ebb-b079-5898a50974f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.243678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e35162f7-164c-4ebb-b079-5898a50974f2 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:31.244864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e35162f7-164c-4ebb-b079-5898a50974f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:31.294916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf505fe-f660-4213-9892-5103f9964b8d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:31.297322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:31.357223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 123/238] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:31.379653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:31.379653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.379653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.379653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.383068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:31.383182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35162f7-164c-4ebb-b079-5898a50974f2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:31.383272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:31.384521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 116/239] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:31.389568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:31.503263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35162f7-164c-4ebb-b079-5898a50974f2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:31.543667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:31.544490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.544795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.546079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.557925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:31.567466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:31.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:31.569279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 124/240] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:31.585085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:31.598410 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76472d5-ad49-45ac-8724-15e7bc4cbcb6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:31.606110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:31.647890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa4955-60f6-4cb5-aa82-2f1ae48a7565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:31.649421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4955-60f6-4cb5-aa82-2f1ae48a7565 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.650558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4955-60f6-4cb5-aa82-2f1ae48a7565 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.651219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4955-60f6-4cb5-aa82-2f1ae48a7565 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.653207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa4955-60f6-4cb5-aa82-2f1ae48a7565 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:31.654950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa4955-60f6-4cb5-aa82-2f1ae48a7565 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:31.657202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:31.693445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 117/241] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 06:59:31 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:31.722095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2104e1-0079-40f0-b634-71345a069012 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.722727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2104e1-0079-40f0-b634-71345a069012 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.723205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2104e1-0079-40f0-b634-71345a069012 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.725300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 118/242] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:31.733717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dab77-c966-484d-a7d1-9847c474f0e3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.735038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dab77-c966-484d-a7d1-9847c474f0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.735521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dab77-c966-484d-a7d1-9847c474f0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.773783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f96dab77-c966-484d-a7d1-9847c474f0e3 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:31.773964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b0c1-bdf9-4078-b204-de1e67dd6f6f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:31.780695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f96dab77-c966-484d-a7d1-9847c474f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:31.816010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:31.820098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:31.820098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:31.820098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:31.820098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:31.820098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:31.820504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:31.840955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 125/243] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:31.857498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:31.885890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 119/244] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:31.911291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dab77-c966-484d-a7d1-9847c474f0e3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:31.971192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:32.001873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:32.003406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d22cc-5485-40c1-84d2-455e5eb22892 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.051224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dab77-c966-484d-a7d1-9847c474f0e3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.056420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.057592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.058086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.058551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] PATH_INFO: `/v3/users/722ecb40638b436b9ab6d3fbfa450630` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.077949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] RBAC: Authorizing `identity:get_user(user_id=722ecb40638b436b9ab6d3fbfa450630)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:32.080506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:32.083696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 126/245] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:31 2022] GET /identity/v3/users/722ecb40638b436b9ab6d3fbfa450630 => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:32.091552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:32.095522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.097141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.097141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.097141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.108751 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:32.116812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:32.118173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:32.118972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 120/246] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:32.132440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:32.226843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c27d23-337e-4a83-8aaf-ca826523e5f0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.266568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.266772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfe2c4-c1cb-465a-80cd-75aacf2a5e59 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.267663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.268170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.268616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.288075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:32.298329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:32.310701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.312069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.312556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.313033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.314004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:32.314589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:32.316310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:32.327706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 121/247] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:32.333719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:32.357004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:32.358605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:32.460722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa579e31-21e5-4513-b926-1ee0fa40a736 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.493129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 127/248] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:32.498409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:32.515474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.517852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.518557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.519719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] PATH_INFO: `/v3/users/0d2613c2bb3848df8cd4f9f2104da2cd` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.522645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] RBAC: Authorizing `identity:get_user(user_id=0d2613c2bb3848df8cd4f9f2104da2cd)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:32.524354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:32.526464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 122/249] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:32 2022] GET /identity/v3/users/0d2613c2bb3848df8cd4f9f2104da2cd => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:32.534287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:32.590805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:32.651070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:32.701977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb580116-ac05-4944-a237-91fcbc5598b9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.747953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93248b9c-0009-44c2-a753-ca7726fe10b5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.766378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f92da-6964-4171-bdaa-0520efcb968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.767670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f92da-6964-4171-bdaa-0520efcb968a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.768149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f92da-6964-4171-bdaa-0520efcb968a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.768632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f92da-6964-4171-bdaa-0520efcb968a None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.770299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f92da-6964-4171-bdaa-0520efcb968a None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:32.771985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f92da-6964-4171-bdaa-0520efcb968a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:32.775313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 128/250] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:32 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:32.777469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57fe35-d720-417f-a34f-c609fe5b2661 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.778129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57fe35-d720-417f-a34f-c609fe5b2661 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.780011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57fe35-d720-417f-a34f-c609fe5b2661 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.782441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 129/251] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:32.784165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57fe35-d720-417f-a34f-c609fe5b2661 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:32.821332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d888b-f344-4ae0-b691-1860f2f04ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.822873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d888b-f344-4ae0-b691-1860f2f04ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.823359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d888b-f344-4ae0-b691-1860f2f04ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.824184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d888b-f344-4ae0-b691-1860f2f04ef2 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.826060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d888b-f344-4ae0-b691-1860f2f04ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:32.828069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d888b-f344-4ae0-b691-1860f2f04ef2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:32.830904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 123/252] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:32 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:32.832317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb86c3e-a1be-4a3d-a526-e5b7fc05efa3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.834444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb86c3e-a1be-4a3d-a526-e5b7fc05efa3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.834975 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb86c3e-a1be-4a3d-a526-e5b7fc05efa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.853596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-afb86c3e-a1be-4a3d-a526-e5b7fc05efa3 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:32.855172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-afb86c3e-a1be-4a3d-a526-e5b7fc05efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:32.945601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57fe35-d720-417f-a34f-c609fe5b2661 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:32.964372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:32.982364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:32.984081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:32.984750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:32.985273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:32.986309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] RBAC: Adding query filter params (role.id=4f981d11e2644f2ea3063f3f139a7bb7, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=722ecb40638b436b9ab6d3fbfa450630) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:32.991922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:32.993694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:33.005841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 130/253] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:32 2022] GET /identity/v3/role_assignments?user.id=722ecb40638b436b9ab6d3fbfa450630&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=4f981d11e2644f2ea3063f3f139a7bb7 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:33.007039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:33.037158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 124/254] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:33.038576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7736e9b-0325-4c89-afbf-a8b7c9a9b17d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:33.038867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7736e9b-0325-4c89-afbf-a8b7c9a9b17d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:33.039145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7736e9b-0325-4c89-afbf-a8b7c9a9b17d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:33.040809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 125/255] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:33.059549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7006728-4656-4a27-8404-06abb13a6bba None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:33.059783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7006728-4656-4a27-8404-06abb13a6bba None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:33.059996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7006728-4656-4a27-8404-06abb13a6bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:33.088571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c7006728-4656-4a27-8404-06abb13a6bba None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:33.090079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c7006728-4656-4a27-8404-06abb13a6bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:33.118943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9deae-3572-4161-b704-b6a5ef5b8a98 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:33.145280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:33.157850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:33.159308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:33.159797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:33.160368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/0d2613c2bb3848df8cd4f9f2104da2cd/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:33.162822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03bdf0b01b44e5abf418124cac9cab, user_id=0d2613c2bb3848df8cd4f9f2104da2cd, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:33.164676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:33.165887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 126/256] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:33.166762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7006728-4656-4a27-8404-06abb13a6bba None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:33.191546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 131/257] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:33 2022] PUT /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/0d2613c2bb3848df8cd4f9f2104da2cd/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:33.235016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:33.306229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7006728-4656-4a27-8404-06abb13a6bba None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:33.312561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:33.346897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:33.347742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:33.348025 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:33.348300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:33.360099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:33.371457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:33.372396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:33.373244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:33.374852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 127/258] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:33.382537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:33.459859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af98508-7654-4efb-b63c-6b4acbbc88d5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:33.485426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:33.548974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:33.549849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:33.550137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:33.550409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:33.582086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:33.584984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67302c3c-c504-4ba3-8574-20d5a856b607 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:33.597278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:33.598219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:33.599029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:33.603938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 132/259] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:33.608837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:33.647776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:33.648949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:33.649558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:33.650023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:33.650985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:33.651485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:33.653356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:33.673208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 128/260] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:33.678379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:33.810747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:33.825280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3959-33fd-47ad-a1a7-34fca2222987 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:33.914378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:33.914992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:33.915285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:33.915558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:33.916324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:33.916635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:33.918085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:33.936573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 133/261] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:33.943302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:33.962967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97a677-f2fb-466e-9971-42e99d92ace7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:33.973423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:34.034717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:34.035620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:34.035903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:34.036178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] PATH_INFO: `/v3/users/386e72b133b242198025d86aeecaad0c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:34.041775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] RBAC: Authorizing `identity:get_user(user_id=386e72b133b242198025d86aeecaad0c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:34.043257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:34.045272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 129/262] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:33 2022] GET /identity/v3/users/386e72b133b242198025d86aeecaad0c => generated 283 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:34.051726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:34.203513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070909-f780-404d-a806-2e9c7603df8b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:34.288473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:34.289401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:34.289687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:34.289967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] PATH_INFO: `/v3/users/386e72b133b242198025d86aeecaad0c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:34.291184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=386e72b133b242198025d86aeecaad0c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:34.292665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:34.299885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 134/263] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:33 2022] GET /identity/v3/users/386e72b133b242198025d86aeecaad0c => generated 283 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:34.300735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:34.301022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ea5cb-1b1d-4525-8f1c-c477f077d88b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:34.352800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:34.391378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:34.392011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:34.392293 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:34.392554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:34.470895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:34.500974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:34.503042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:34.507311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1907e0d-3b8d-428f-abef-02e92a376eb6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:34.581923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:34.582589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:34.582867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:34.583141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:34.700498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:34.702665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:34.704078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 130/264] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:34.728706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:34.739016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:34.825483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 135/265] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 06:59:34.832859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:34.903114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:34.948357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f25c60-fa59-4300-9cf4-36722e871fbb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:34.975250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb243a3-5121-43e4-b135-e4d7d867fb88 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:34.979337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:34.998505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7cbb0-c6ac-4145-ac4a-f0a6a0ad965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:34.999838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cbb0-c6ac-4145-ac4a-f0a6a0ad965c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.000433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cbb0-c6ac-4145-ac4a-f0a6a0ad965c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.000927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cbb0-c6ac-4145-ac4a-f0a6a0ad965c None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.003546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7cbb0-c6ac-4145-ac4a-f0a6a0ad965c None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:35.007799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7cbb0-c6ac-4145-ac4a-f0a6a0ad965c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:35.009834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 136/266] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:34 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:35.018878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569014c-8bbe-482e-8bae-8157a0b66616 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.020766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569014c-8bbe-482e-8bae-8157a0b66616 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.021345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569014c-8bbe-482e-8bae-8157a0b66616 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.022928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 137/267] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:35.025298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369822db-c469-417f-a3cd-40d0236534a8 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.025775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369822db-c469-417f-a3cd-40d0236534a8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.026220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369822db-c469-417f-a3cd-40d0236534a8 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.027834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 138/268] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:35.032576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369822db-c469-417f-a3cd-40d0236534a8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.042102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b8632-866a-4133-96a6-d0934535f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:35.042974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b8632-866a-4133-96a6-d0934535f68d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.043503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b8632-866a-4133-96a6-d0934535f68d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.044006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b8632-866a-4133-96a6-d0934535f68d None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.045424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b8632-866a-4133-96a6-d0934535f68d None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:35.047236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b8632-866a-4133-96a6-d0934535f68d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:35.049489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 131/269] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:34 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:35.050870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f72f2-c783-49eb-95c0-91a16f7f3a85 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.051359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f72f2-c783-49eb-95c0-91a16f7f3a85 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.051821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f72f2-c783-49eb-95c0-91a16f7f3a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.070649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-720f72f2-c783-49eb-95c0-91a16f7f3a85 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:35.072203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-720f72f2-c783-49eb-95c0-91a16f7f3a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:35.145688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 132/270] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:35.147509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4659fda-a009-4209-811e-c18e8c9c6dc8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.147991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4659fda-a009-4209-811e-c18e8c9c6dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.148454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4659fda-a009-4209-811e-c18e8c9c6dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.157721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369822db-c469-417f-a3cd-40d0236534a8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:35.170153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f4659fda-a009-4209-811e-c18e8c9c6dc8 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:35.172439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f4659fda-a009-4209-811e-c18e8c9c6dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:35.203416 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:35.204319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.205258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.205700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.206727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] RBAC: Adding query filter params (role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=386e72b133b242198025d86aeecaad0c) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:35.207208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:35.208812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:35.215545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 139/271] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:35 2022] GET /identity/v3/role_assignments?user.id=386e72b133b242198025d86aeecaad0c&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:35.216526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.287130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 133/272] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:35.288529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4659fda-a009-4209-811e-c18e8c9c6dc8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.292579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:35.341065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:35.427088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff58767-17ea-4ac5-ab39-ef2292220e24 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:35.470874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:35.471931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.472409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.472877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.473612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4659fda-a009-4209-811e-c18e8c9c6dc8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:35.474522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=386e72b133b242198025d86aeecaad0c) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:35.475029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:35.476685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:35.489471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 140/273] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:35 2022] GET /identity/v3/role_assignments?user.id=386e72b133b242198025d86aeecaad0c&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:35.490778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.515801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:35.517086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.517511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.517923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.532600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:35.542311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:35.543259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:35.544069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:35.547190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 134/274] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:35.558364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.694019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:35.706585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d3233-0319-4e7f-aae3-621abc73e93b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:35.732580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:35.740092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590a510-b2c9-4c0a-a01d-975c61f3f962 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:35.748112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4ad74-6f49-40ef-a5af-f88e0b156cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:35.748927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ad74-6f49-40ef-a5af-f88e0b156cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.749296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ad74-6f49-40ef-a5af-f88e0b156cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.749576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ad74-6f49-40ef-a5af-f88e0b156cb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.750868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4ad74-6f49-40ef-a5af-f88e0b156cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:35.752322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4ad74-6f49-40ef-a5af-f88e0b156cb4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:35.756966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 141/275] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:35.756966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f25ca5-ec8e-4463-9473-109b09418dd0 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.756966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f25ca5-ec8e-4463-9473-109b09418dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.756966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f25ca5-ec8e-4463-9473-109b09418dd0 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.757449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 142/276] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:35.762593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f25ca5-ec8e-4463-9473-109b09418dd0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.785994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba686548-ee75-47d6-96a0-b6529e78d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:35.786680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba686548-ee75-47d6-96a0-b6529e78d8fa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.786965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba686548-ee75-47d6-96a0-b6529e78d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.787240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba686548-ee75-47d6-96a0-b6529e78d8fa None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.788004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba686548-ee75-47d6-96a0-b6529e78d8fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:35.788318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba686548-ee75-47d6-96a0-b6529e78d8fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:35.789763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba686548-ee75-47d6-96a0-b6529e78d8fa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:35.799745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 135/277] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:35.802723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3438c-62ee-430c-90ba-4b4ff9c9045c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.803083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3438c-62ee-430c-90ba-4b4ff9c9045c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.803431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3438c-62ee-430c-90ba-4b4ff9c9045c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.829773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d7a3438c-62ee-430c-90ba-4b4ff9c9045c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:35.830987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d7a3438c-62ee-430c-90ba-4b4ff9c9045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:35.885560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f25ca5-ec8e-4463-9473-109b09418dd0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:35.913203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 136/278] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:35.914119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3438c-62ee-430c-90ba-4b4ff9c9045c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.925956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:35.926706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:35.926997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:35.927276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:35.928447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:35.930279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:35.953586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 143/279] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 06:59:35 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 06:59:35.954806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:35.984727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:36.035871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:36.068245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3438c-62ee-430c-90ba-4b4ff9c9045c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.106625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d91213-00cf-4590-b967-9c3e0203e9da None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.108459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff8e1a-20d7-4311-bbff-42f0a7aec099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.109786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff8e1a-20d7-4311-bbff-42f0a7aec099 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.110073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff8e1a-20d7-4311-bbff-42f0a7aec099 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.110346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff8e1a-20d7-4311-bbff-42f0a7aec099 None admin] PATH_INFO: `/v3/users/29b9031cc7e046e698b9ba3dee793257` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.128349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff8e1a-20d7-4311-bbff-42f0a7aec099 None admin] RBAC: Authorizing `identity:get_user(user_id=29b9031cc7e046e698b9ba3dee793257)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:36.129821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff8e1a-20d7-4311-bbff-42f0a7aec099 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:36.132296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 137/280] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:35 2022] GET /identity/v3/users/29b9031cc7e046e698b9ba3dee793257 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:36.133990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9316a1-96ae-4113-b070-5140a26df777 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.134188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9316a1-96ae-4113-b070-5140a26df777 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.134473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9316a1-96ae-4113-b070-5140a26df777 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.135882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 138/281] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:36.137870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9316a1-96ae-4113-b070-5140a26df777 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:36.150749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d05f0-0ee3-4277-8b7a-93e2b9dce9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.151851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d05f0-0ee3-4277-8b7a-93e2b9dce9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.152134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d05f0-0ee3-4277-8b7a-93e2b9dce9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.152413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d05f0-0ee3-4277-8b7a-93e2b9dce9d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.165337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d05f0-0ee3-4277-8b7a-93e2b9dce9d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-237d05f0-0ee3-4277-8b7a-93e2b9dce9d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:36.173212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:36.174127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:36.174925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 144/282] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:36.175803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650150f5-a6a2-445e-b1a5-78ca8e3b01e5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.176092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650150f5-a6a2-445e-b1a5-78ca8e3b01e5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.176374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650150f5-a6a2-445e-b1a5-78ca8e3b01e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.194158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-650150f5-a6a2-445e-b1a5-78ca8e3b01e5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:36.195372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-650150f5-a6a2-445e-b1a5-78ca8e3b01e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:36.264875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9316a1-96ae-4113-b070-5140a26df777 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.276978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 145/283] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:36.277873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650150f5-a6a2-445e-b1a5-78ca8e3b01e5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:36.281933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:36.318075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.319260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.319955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.320656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.321721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, user.id=29b9031cc7e046e698b9ba3dee793257, scope.system=all) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:36.322201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:36.325269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:36.332503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 139/284] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 06:59:36 2022] GET /identity/v3/role_assignments?user.id=29b9031cc7e046e698b9ba3dee793257&scope.system=all&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:36.333889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:36.335257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:36.458821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27614bf-29f9-4bd9-84cd-ed642f3a6382 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.499317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.500837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.501277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.501961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.515063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:36.520679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650150f5-a6a2-445e-b1a5-78ca8e3b01e5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:36.525292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:36.526442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:36.527431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:36.529667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 140/285] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:36.541991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:36.575278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.575890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.576171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.576444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.577350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:36.577681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:36.579569 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:36.590222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:36.596166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 146/286] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:36.602203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:36.713267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa0991-8973-4fde-8c40-5588eb0ebd4a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.721397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:36.750944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.751838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.752119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.752391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] PATH_INFO: `/v3/users/722ecb40638b436b9ab6d3fbfa450630` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.753631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] RBAC: Authorizing `identity:get_user(user_id=722ecb40638b436b9ab6d3fbfa450630)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:36.755623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:36.757637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 147/287] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:36 2022] GET /identity/v3/users/722ecb40638b436b9ab6d3fbfa450630 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:36.763195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:36.779967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ff2e2-f349-4c42-8aba-33d05d3cdcf0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.833523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.834125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.834408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.834678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.835435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:36.835741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:36.837191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:36.849179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 141/288] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:36.851924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:36.897545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3e079-7be0-4d10-9280-e1433697358a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.954813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:36.958783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f82cb-60a4-4607-a9c0-302d8eadc0a5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:36.973873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:36.974774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:36.975051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:36.975323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:36.976753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:36.978285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:36.980429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 148/289] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:36 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:36.985945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:37.005812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:37.006762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:37.013072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:37.013072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] PATH_INFO: `/v3/users/0d2613c2bb3848df8cd4f9f2104da2cd` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:37.014061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] RBAC: Authorizing `identity:get_user(user_id=0d2613c2bb3848df8cd4f9f2104da2cd)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:37.015758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:37.022220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 142/290] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:36 2022] GET /identity/v3/users/0d2613c2bb3848df8cd4f9f2104da2cd => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:37.023705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:37.063470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:37.191904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39eae9-b00f-4ef4-838f-cceba1110676 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:37.205933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d2f5b-f4d3-46f2-94d4-074c9fbc155d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:37.241696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:37.242354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:37.242639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:37.242751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:37.242938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:37.244111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:37.244580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:37.245069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/722ecb40638b436b9ab6d3fbfa450630/roles/4f981d11e2644f2ea3063f3f139a7bb7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:37.248181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03bdf0b01b44e5abf418124cac9cab, user_id=722ecb40638b436b9ab6d3fbfa450630, role_id=4f981d11e2644f2ea3063f3f139a7bb7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:37.249913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:37.282940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 149/291] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 06:59:36 2022] PUT /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/users/722ecb40638b436b9ab6d3fbfa450630/roles/4f981d11e2644f2ea3063f3f139a7bb7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:37.289481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:37.321266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:37.322974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:37.439792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 143/292] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:37.441301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:37.592737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:37.602008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365b568-a612-46e7-bcf2-3ebb34b62db5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:37.639076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f16f6-d921-41a3-87d8-210b63b2f120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:37.640097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f16f6-d921-41a3-87d8-210b63b2f120 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:37.640437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f16f6-d921-41a3-87d8-210b63b2f120 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:37.640709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f16f6-d921-41a3-87d8-210b63b2f120 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:37.641931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f16f6-d921-41a3-87d8-210b63b2f120 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:37.643378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f16f6-d921-41a3-87d8-210b63b2f120 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:37.645434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 150/293] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:37 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:37.651563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:37.746766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:37.791817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6b0a-4523-4ede-a8ab-0de4b2cfa545 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:37.844978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d670-02da-4980-aa1a-3a4d46f3ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:37.845593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d670-02da-4980-aa1a-3a4d46f3ae73 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:37.845875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d670-02da-4980-aa1a-3a4d46f3ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:37.846149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d670-02da-4980-aa1a-3a4d46f3ae73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:37.846992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d670-02da-4980-aa1a-3a4d46f3ae73 None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=0d2613c2bb3848df8cd4f9f2104da2cd) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:37.847296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d670-02da-4980-aa1a-3a4d46f3ae73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:37.848761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d670-02da-4980-aa1a-3a4d46f3ae73 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:37.861203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 144/294] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:37 2022] GET /identity/v3/role_assignments?user.id=0d2613c2bb3848df8cd4f9f2104da2cd&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:38.372578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded056d-09b2-4e1e-88bc-77773901ffa4 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.373143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded056d-09b2-4e1e-88bc-77773901ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.373143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded056d-09b2-4e1e-88bc-77773901ffa4 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.374826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 151/295] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:38.382137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1f97-2f00-4764-b069-dd4abd197563 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.382485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1f97-2f00-4764-b069-dd4abd197563 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.382770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1f97-2f00-4764-b069-dd4abd197563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.409228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e71a1f97-2f00-4764-b069-dd4abd197563 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:38.410485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e71a1f97-2f00-4764-b069-dd4abd197563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:38.430505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116af35-2187-4060-ae15-6936db17963c None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.430807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116af35-2187-4060-ae15-6936db17963c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.431060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116af35-2187-4060-ae15-6936db17963c None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.433764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 152/296] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:38.437849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f6f32-9667-42f9-bc5f-eb2f708b3a79 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.438134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f6f32-9667-42f9-bc5f-eb2f708b3a79 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.438414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f6f32-9667-42f9-bc5f-eb2f708b3a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.457883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f20f6f32-9667-42f9-bc5f-eb2f708b3a79 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:38.459057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f20f6f32-9667-42f9-bc5f-eb2f708b3a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:38.492011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 145/297] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:38.515572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1f97-2f00-4764-b069-dd4abd197563 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:38.529141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 153/298] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:38.551611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f6f32-9667-42f9-bc5f-eb2f708b3a79 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:38.586079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:38.637650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1f97-2f00-4764-b069-dd4abd197563 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:38.661086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:38.672971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f6f32-9667-42f9-bc5f-eb2f708b3a79 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:38.673572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:38.674397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.674679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.674950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.677761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:38.678797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:38.680473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 146/299] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:38.685970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:38.723668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:38.724736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.725016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.725312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.740243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:38.750964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:38.751885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:38.752695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 154/300] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:38.758134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:38.822675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f3108-63c5-4aa3-8164-efa63b8492f3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:38.852219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:38.864717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:38.865997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.866428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.866864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] PATH_INFO: `/v3/groups` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.867705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:38.870106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:38.881787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e15cfe-cc2b-4934-af12-d2cd2c05cda2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:38.894068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 147/301] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Jul 12 06:59:38 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 06:59:38.927653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac88f7f-2fb4-4947-9241-47c3ee6567ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:38.928449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac88f7f-2fb4-4947-9241-47c3ee6567ca None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:38.928796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac88f7f-2fb4-4947-9241-47c3ee6567ca None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:38.929166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac88f7f-2fb4-4947-9241-47c3ee6567ca None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:38.930087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac88f7f-2fb4-4947-9241-47c3ee6567ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:38.930498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac88f7f-2fb4-4947-9241-47c3ee6567ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:38.932060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac88f7f-2fb4-4947-9241-47c3ee6567ca None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:38.942838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 155/302] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:38.949381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:39.054400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756012f-ef53-4f5a-9ff6-87e4acc0e02b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.054400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756012f-ef53-4f5a-9ff6-87e4acc0e02b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.054400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756012f-ef53-4f5a-9ff6-87e4acc0e02b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.054400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 156/303] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:39.059138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93423-916b-4521-8261-5dfe6b8512d4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.059573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93423-916b-4521-8261-5dfe6b8512d4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.060005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93423-916b-4521-8261-5dfe6b8512d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.069452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ae43b-476e-418f-aaaa-91d63bb46624 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:39.084934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f2e93423-916b-4521-8261-5dfe6b8512d4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:39.086559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f2e93423-916b-4521-8261-5dfe6b8512d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:39.100426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:39.117096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:39.117977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.118806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.118806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] PATH_INFO: `/v3/users/aa0b32b5e1e241e8b1869bd458a4e2b6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.134557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] RBAC: Authorizing `identity:get_user(user_id=aa0b32b5e1e241e8b1869bd458a4e2b6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:39.136040 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:39.142737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 148/304] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:38 2022] GET /identity/v3/users/aa0b32b5e1e241e8b1869bd458a4e2b6 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:39.143987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:39.179932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 157/305] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:39.199734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:39.204043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e93423-916b-4521-8261-5dfe6b8512d4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:39.263499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6d77a-b526-4d07-9dfe-6564f6810a71 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:39.304452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:39.306063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.306601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.307077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.307929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e93423-916b-4521-8261-5dfe6b8512d4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:39.308273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] RBAC: Adding query filter params (role.id=4f981d11e2644f2ea3063f3f139a7bb7, user.id=aa0b32b5e1e241e8b1869bd458a4e2b6, scope.system=all) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:39.308820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:39.311852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:39.319054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 149/306] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 06:59:39 2022] GET /identity/v3/role_assignments?user.id=aa0b32b5e1e241e8b1869bd458a4e2b6&scope.system=all&role.id=4f981d11e2644f2ea3063f3f139a7bb7 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:39.354088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:39.354935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.355216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.355494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.368257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:39.376687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:39.378113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:39.378902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 158/307] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:39.385237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:39.495107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47b66-0c79-4ba3-af13-39f6c2ef8679 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:39.506118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:39.530140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509e41c-7b61-424c-aaee-20acbbfe5489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:39.530880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509e41c-7b61-424c-aaee-20acbbfe5489 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.531163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509e41c-7b61-424c-aaee-20acbbfe5489 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.531436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509e41c-7b61-424c-aaee-20acbbfe5489 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.532198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509e41c-7b61-424c-aaee-20acbbfe5489 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:39.532512 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509e41c-7b61-424c-aaee-20acbbfe5489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:39.533931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509e41c-7b61-424c-aaee-20acbbfe5489 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:39.542719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 150/308] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:39.550213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:39.556710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:39.659558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334826e9-e74e-468f-9594-6ade52464eb1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.659887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334826e9-e74e-468f-9594-6ade52464eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.660112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334826e9-e74e-468f-9594-6ade52464eb1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.661810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 151/309] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:39.666568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72fd51-9fa3-4a51-ae0b-c85ca0677ca8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:39.666898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d57e51-935b-40b6-8be8-3b8dcbcb4f4b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.667194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d57e51-935b-40b6-8be8-3b8dcbcb4f4b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.667480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d57e51-935b-40b6-8be8-3b8dcbcb4f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.687882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-10d57e51-935b-40b6-8be8-3b8dcbcb4f4b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:39.689108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-10d57e51-935b-40b6-8be8-3b8dcbcb4f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:39.705281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:39.706264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.706611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.706944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] PATH_INFO: `/v3/users/29b9031cc7e046e698b9ba3dee793257` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.708211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] RBAC: Authorizing `identity:get_user(user_id=29b9031cc7e046e698b9ba3dee793257)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:39.709786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:39.711801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 159/310] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:39 2022] GET /identity/v3/users/29b9031cc7e046e698b9ba3dee793257 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:39.717988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:39.773621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 152/311] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:39.796996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d57e51-935b-40b6-8be8-3b8dcbcb4f4b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:39.806422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:39.836103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c38e33-2f5d-48db-aa8f-daa7a124a8f8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:39.876511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:39.877438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.877711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.877999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] PATH_INFO: `/v3/system/users/29b9031cc7e046e698b9ba3dee793257/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.879768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=29b9031cc7e046e698b9ba3dee793257, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:39.880856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:39.894938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 160/312] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Jul 12 06:59:39 2022] PUT /identity/v3/system/users/29b9031cc7e046e698b9ba3dee793257/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:39.924328 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d57e51-935b-40b6-8be8-3b8dcbcb4f4b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:39.926616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:39.970264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:39.971098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:39.971378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:39.971651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:39.984464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:40.004223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:40.005309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:40.006110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 153/313] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:40.011738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:40.117476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1614a74-907c-4fea-bb71-300a4cf58a52 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:40.151109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:40.151785 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:40.152065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:40.152385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:40.153167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:40.153475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:40.154876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:40.157240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:40.164091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 161/314] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:40.169739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:40.277084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7c70-b312-4fc9-9e88-0ff98c0fd0ca None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:40.312111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:40.313071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:40.313355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:40.313678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] PATH_INFO: `/v3/users/722ecb40638b436b9ab6d3fbfa450630` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:40.315009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] RBAC: Authorizing `identity:get_user(user_id=722ecb40638b436b9ab6d3fbfa450630)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:40.316468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:40.318465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 154/315] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:40 2022] GET /identity/v3/users/722ecb40638b436b9ab6d3fbfa450630 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:40.324355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:40.433031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54679ac6-374c-4d1c-9af6-e3f3cdc4b5d3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:40.477140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:40.478649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:40.479325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:40.480009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:40.550745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:40.555131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:40.556794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:40.659639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 162/316] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:40.665587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:40.699080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:40.776903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ec9af-d05c-4000-8ece-380450e2c2af None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:40.812249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48700ed-7670-4dcd-9d81-4dfa1d3a4491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:40.813153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48700ed-7670-4dcd-9d81-4dfa1d3a4491 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:40.813437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48700ed-7670-4dcd-9d81-4dfa1d3a4491 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:40.813710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48700ed-7670-4dcd-9d81-4dfa1d3a4491 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:40.814903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48700ed-7670-4dcd-9d81-4dfa1d3a4491 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:40.816371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48700ed-7670-4dcd-9d81-4dfa1d3a4491 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:40.818441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 155/317] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:40 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:40.824473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:40.924568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:40.928733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd304ecc-d12b-4ffb-9bff-1c578cf3cc7b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:40.964346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4a8b6-b18e-4a52-bb5f-2b0b7e03a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:40.964925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a8b6-b18e-4a52-bb5f-2b0b7e03a199 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:40.965234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a8b6-b18e-4a52-bb5f-2b0b7e03a199 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:40.965512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a8b6-b18e-4a52-bb5f-2b0b7e03a199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:40.966345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4a8b6-b18e-4a52-bb5f-2b0b7e03a199 None admin] RBAC: Adding query filter params (role.id=4f981d11e2644f2ea3063f3f139a7bb7, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab, user.id=722ecb40638b436b9ab6d3fbfa450630) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:40.966657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4a8b6-b18e-4a52-bb5f-2b0b7e03a199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:40.968298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4a8b6-b18e-4a52-bb5f-2b0b7e03a199 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:40.980323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 163/318] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 06:59:40 2022] GET /identity/v3/role_assignments?user.id=722ecb40638b436b9ab6d3fbfa450630&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=4f981d11e2644f2ea3063f3f139a7bb7 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:41.205448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd795382-c98d-47b0-b52b-5ca49218dffe None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.206359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd795382-c98d-47b0-b52b-5ca49218dffe None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.206849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd795382-c98d-47b0-b52b-5ca49218dffe None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.208446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 156/319] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:41.213902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc3ca4-7b5c-4724-a4c9-38e343d17481 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.214492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc3ca4-7b5c-4724-a4c9-38e343d17481 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.214969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc3ca4-7b5c-4724-a4c9-38e343d17481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.233319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bedc3ca4-7b5c-4724-a4c9-38e343d17481 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:41.235104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bedc3ca4-7b5c-4724-a4c9-38e343d17481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:41.304463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 164/320] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:41.326981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd795382-c98d-47b0-b52b-5ca49218dffe None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:41.425737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:41.426967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c741d0-908f-405c-82df-088b4574931b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.427360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c741d0-908f-405c-82df-088b4574931b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.427526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c741d0-908f-405c-82df-088b4574931b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.429409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 165/321] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:41.432197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd795382-c98d-47b0-b52b-5ca49218dffe None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:41.434337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3eea4-a66b-4b15-98fb-73d75ed7dcd9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.434801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3eea4-a66b-4b15-98fb-73d75ed7dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.435021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3eea4-a66b-4b15-98fb-73d75ed7dcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.452669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5ed3eea4-a66b-4b15-98fb-73d75ed7dcd9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:41.453873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5ed3eea4-a66b-4b15-98fb-73d75ed7dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:41.468341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:41.469266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.469546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.469857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.470957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:41.472542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:41.474226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 157/322] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 06:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:41.479467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:41.533388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 166/323] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:41.547495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3eea4-a66b-4b15-98fb-73d75ed7dcd9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:41.575124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:41.582658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408999c2-73db-4da3-bdab-98d9c18d2d94 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:41.618661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:41.619729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.620013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.620287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] PATH_INFO: `/v3/groups` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.620981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:41.622455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:41.637871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 158/324] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Jul 12 06:59:41 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 06:59:41.663975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3eea4-a66b-4b15-98fb-73d75ed7dcd9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:41.704321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:41.705172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.705451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.705709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.717709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:41.726022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:41.727024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:41.733312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 167/325] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:41.735578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:41.767848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:41.853287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd48cf4-f00d-4b96-ba5a-6c9dec9ee71a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:41.896129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dab676-fb74-4c21-bab4-e9c3486aca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:41.896811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dab676-fb74-4c21-bab4-e9c3486aca8b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:41.897108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dab676-fb74-4c21-bab4-e9c3486aca8b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:41.897384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dab676-fb74-4c21-bab4-e9c3486aca8b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:41.898135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dab676-fb74-4c21-bab4-e9c3486aca8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:41.898456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dab676-fb74-4c21-bab4-e9c3486aca8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:41.899816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dab676-fb74-4c21-bab4-e9c3486aca8b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:41.910316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 159/326] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:41.916204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:42.018469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:42.037451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d4c3-f812-4232-9e41-779ca7ea1dfb None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.038060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d4c3-f812-4232-9e41-779ca7ea1dfb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.038512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ca281-8c81-42cb-9d23-8e361f83155f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:42.038788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949d4c3-f812-4232-9e41-779ca7ea1dfb None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.040377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 160/327] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:42.045740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a4f3-46c5-4d78-8a11-ecd5bb838c50 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.046420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a4f3-46c5-4d78-8a11-ecd5bb838c50 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.046887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a4f3-46c5-4d78-8a11-ecd5bb838c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.072969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-50f8a4f3-46c5-4d78-8a11-ecd5bb838c50 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:42.074623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-50f8a4f3-46c5-4d78-8a11-ecd5bb838c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:42.078285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:42.079271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.079557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.079834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] PATH_INFO: `/v3/users/aa0b32b5e1e241e8b1869bd458a4e2b6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.081845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] RBAC: Authorizing `identity:get_user(user_id=aa0b32b5e1e241e8b1869bd458a4e2b6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:42.083543 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:42.085815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 168/328] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:41 2022] GET /identity/v3/users/aa0b32b5e1e241e8b1869bd458a4e2b6 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:42.091035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:42.162696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 161/329] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:42.188951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a4f3-46c5-4d78-8a11-ecd5bb838c50 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:42.218949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:42.244413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc3489-915c-4e81-89c0-b5e09dc7da45 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:42.311569 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:42.314109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.314109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.314109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] PATH_INFO: `/v3/system/users/aa0b32b5e1e241e8b1869bd458a4e2b6/roles/4f981d11e2644f2ea3063f3f139a7bb7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.315586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aa0b32b5e1e241e8b1869bd458a4e2b6, role_id=4f981d11e2644f2ea3063f3f139a7bb7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:42.316956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:42.318695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a4f3-46c5-4d78-8a11-ecd5bb838c50 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:42.326658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 169/330] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Jul 12 06:59:42 2022] PUT /identity/v3/system/users/aa0b32b5e1e241e8b1869bd458a4e2b6/roles/4f981d11e2644f2ea3063f3f139a7bb7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 06:59:42.358401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:42.359736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.360242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.360721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.373678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:42.382123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:42.383106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:42.383897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:42.386721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 162/331] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:42.393775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:42.428169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:42.542474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a1f60-4837-47e0-99ae-ecf7cc597f41 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:42.580962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:42.581677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.581961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.582239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.583004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:42.583314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:42.584791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:42.593862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 170/332] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:42.599573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:42.693835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:42.720683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f77d95-076b-4a0e-b3c4-6df6dc10782a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:42.755921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414614ad-3fa5-47a3-87fa-74a0ce532c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:42.756817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414614ad-3fa5-47a3-87fa-74a0ce532c68 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.757122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414614ad-3fa5-47a3-87fa-74a0ce532c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.757399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414614ad-3fa5-47a3-87fa-74a0ce532c68 None admin] PATH_INFO: `/v3/users/29b9031cc7e046e698b9ba3dee793257` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.758600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414614ad-3fa5-47a3-87fa-74a0ce532c68 None admin] RBAC: Authorizing `identity:get_user(user_id=29b9031cc7e046e698b9ba3dee793257)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:42.760073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414614ad-3fa5-47a3-87fa-74a0ce532c68 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:42.762044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 163/333] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:42 2022] GET /identity/v3/users/29b9031cc7e046e698b9ba3dee793257 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:42.768616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:42.867048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:42.891109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62566167-315f-4e99-bf18-aa0461c9e418 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:42.938928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ff7bb-e0d4-4981-9612-736284d8501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:42.939603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ff7bb-e0d4-4981-9612-736284d8501a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:42.939886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ff7bb-e0d4-4981-9612-736284d8501a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:42.940167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ff7bb-e0d4-4981-9612-736284d8501a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:42.941008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ff7bb-e0d4-4981-9612-736284d8501a None admin] RBAC: Adding query filter params (role.id=9b4fe27dba204b05b692207de4fcedd4, user.id=29b9031cc7e046e698b9ba3dee793257, scope.system=all) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:42.941342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ff7bb-e0d4-4981-9612-736284d8501a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:42.942797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ff7bb-e0d4-4981-9612-736284d8501a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:42.959254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 171/334] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 06:59:42 2022] GET /identity/v3/role_assignments?user.id=29b9031cc7e046e698b9ba3dee793257&scope.system=all&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:44.237423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29549-8a45-4339-8918-07f05d296fc7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:44.238438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29549-8a45-4339-8918-07f05d296fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:44.238917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29549-8a45-4339-8918-07f05d296fc7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:44.240797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 164/335] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:44.246109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc679e-1cf6-40ba-8d92-618395d549ef None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:44.246944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc679e-1cf6-40ba-8d92-618395d549ef None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:44.247413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc679e-1cf6-40ba-8d92-618395d549ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:44.266835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-58cc679e-1cf6-40ba-8d92-618395d549ef None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:44.268389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-58cc679e-1cf6-40ba-8d92-618395d549ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:44.345594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 172/336] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:44.373309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29549-8a45-4339-8918-07f05d296fc7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:44.463051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21625cb9-b95d-4c76-a2a5-357f91cc3c83 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:44.463688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21625cb9-b95d-4c76-a2a5-357f91cc3c83 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:44.464181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21625cb9-b95d-4c76-a2a5-357f91cc3c83 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:44.466690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 173/337] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:44.473151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fefc35-ebdf-43ac-a5e0-5b161fffe7dd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:44.473860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fefc35-ebdf-43ac-a5e0-5b161fffe7dd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:44.474438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fefc35-ebdf-43ac-a5e0-5b161fffe7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:44.494504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-63fefc35-ebdf-43ac-a5e0-5b161fffe7dd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:44.496707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-63fefc35-ebdf-43ac-a5e0-5b161fffe7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:44.511314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29549-8a45-4339-8918-07f05d296fc7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:44.575409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:44.582367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:44.583213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:44.583481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:44.583759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:44.586501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 174/338] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:44.601928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:44.610709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:44.611619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:44.612412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:44.615673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fefc35-ebdf-43ac-a5e0-5b161fffe7dd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:44.617080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 165/339] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:44.620661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:44.631657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:44.803213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fefc35-ebdf-43ac-a5e0-5b161fffe7dd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:44.858428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d3129-0cbe-4f0a-b01b-0bca3c07e1b8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:44.882837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:44.883665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:44.883947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:44.884221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:44.914038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:44.922729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:44.923656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:44.924453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:44.927575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 175/340] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:44.932460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:44.932460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:44.932460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:44.932460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:44.932803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:44.932803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:44.932803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:44.934104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:44.951220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 166/341] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:44.957334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:45.048155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:45.109316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafc65c-8e7a-4215-9869-4aba8f14f357 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:45.137402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:45.145793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19febd4a-19ec-4b70-ab88-14b90591cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:45.146734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19febd4a-19ec-4b70-ab88-14b90591cfaa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.147195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19febd4a-19ec-4b70-ab88-14b90591cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.147608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19febd4a-19ec-4b70-ab88-14b90591cfaa None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.148523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19febd4a-19ec-4b70-ab88-14b90591cfaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:45.148977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19febd4a-19ec-4b70-ab88-14b90591cfaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:45.150589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19febd4a-19ec-4b70-ab88-14b90591cfaa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:45.163878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 176/342] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:45.165707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4422ac-efe7-40e1-ba09-d72721cfef75 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.166139 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4422ac-efe7-40e1-ba09-d72721cfef75 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.166554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4422ac-efe7-40e1-ba09-d72721cfef75 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.168355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 177/343] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:45.174083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230762e-987b-4048-958e-6f619d299bec None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.174586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230762e-987b-4048-958e-6f619d299bec None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.175004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230762e-987b-4048-958e-6f619d299bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.195041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2230762e-987b-4048-958e-6f619d299bec None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:45.196517 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2230762e-987b-4048-958e-6f619d299bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:45.216791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc00f-4885-4ca5-b978-8c58392d44f5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:45.268830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 178/344] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 12 06:59:45.270334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230762e-987b-4048-958e-6f619d299bec None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:45.306941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:45.307905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.308356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.308767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.339198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:45.408065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230762e-987b-4048-958e-6f619d299bec None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:45.432138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:45.438976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:45.443690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d245eb0-9581-4034-ac15-7602719442b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:45.444622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d245eb0-9581-4034-ac15-7602719442b4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.445133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d245eb0-9581-4034-ac15-7602719442b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.445577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d245eb0-9581-4034-ac15-7602719442b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.549479 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:45.556171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d245eb0-9581-4034-ac15-7602719442b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:45.557928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d245eb0-9581-4034-ac15-7602719442b4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:45.603476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 167/345] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 06:59:45.604679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:45.625834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:45.705535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 179/346] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:45.709177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5d21-9ecc-428c-bc99-cab521a6fefb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.709627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5d21-9ecc-428c-bc99-cab521a6fefb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.710182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5d21-9ecc-428c-bc99-cab521a6fefb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.715674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 180/347] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:45.716629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa5d21-9ecc-428c-bc99-cab521a6fefb None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:45.744114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0adbc-4070-4a28-b7aa-816d1ee467f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:45.789572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:45.790890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.791312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.791961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.806820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:45.816304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:45.817584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:45.818582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:45.820936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 168/348] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:45.821877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:45.850068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa5d21-9ecc-428c-bc99-cab521a6fefb None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:45.863551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:45.899955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070cc67-4562-4d77-9115-9a5691ced4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:45.901539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070cc67-4562-4d77-9115-9a5691ced4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.902205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070cc67-4562-4d77-9115-9a5691ced4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.902620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070cc67-4562-4d77-9115-9a5691ced4a8 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.904630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070cc67-4562-4d77-9115-9a5691ced4a8 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:45.906736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070cc67-4562-4d77-9115-9a5691ced4a8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:45.908954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 181/349] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:45 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:45.910298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d380e-5290-4e51-8bd3-1e4d3048382d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.910708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d380e-5290-4e51-8bd3-1e4d3048382d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.911103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d380e-5290-4e51-8bd3-1e4d3048382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:45.932656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-152d380e-5290-4e51-8bd3-1e4d3048382d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:45.934332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-152d380e-5290-4e51-8bd3-1e4d3048382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:45.948064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:45.955117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bae314-9a34-4b3c-bc43-de3e3b660a2c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:45.997353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:45.998591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:45.999131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:45.999821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.001348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:46.008891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:46.008891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 169/350] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:45 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:46.008891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.027280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 182/351] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:46.028465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d380e-5290-4e51-8bd3-1e4d3048382d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.151072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d380e-5290-4e51-8bd3-1e4d3048382d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:46.167870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:46.174723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8c73e-ba4c-4eef-a67a-86bf845ea6ec None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:46.204038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:46.205559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.206043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.206523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.217668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:46.219943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:46.220808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:46.221542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.221824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.222098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.222877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 183/352] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:46.222979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:46.223911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.228468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:46.229478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:46.238679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:46.245385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 170/353] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:46.246391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.359220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c5ed2-4d45-4139-b436-ed28528f1276 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:46.372806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3beff-07b2-436a-b0e7-29c9952fc756 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:46.398779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:46.399655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.399936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.400211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.402328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:46.403794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:46.405487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 184/354] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:46.406281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.415591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:46.416468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.416753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.418125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.432093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:46.440393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:46.441383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:46.442183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 171/355] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:46.442853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.464262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:46.515636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:46.548551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b34f31-2087-4691-9d5e-e779a592b5b9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:46.573172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651017d-b9aa-40d1-be60-65a5b5c8a3f5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:46.587648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccd069-857e-4888-86a9-6b1b9a03e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:46.588356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccd069-857e-4888-86a9-6b1b9a03e061 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.588640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccd069-857e-4888-86a9-6b1b9a03e061 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.588914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccd069-857e-4888-86a9-6b1b9a03e061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.590022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccd069-857e-4888-86a9-6b1b9a03e061 None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=19009ab53c574545b66af5787e770785) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:46.590128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccd069-857e-4888-86a9-6b1b9a03e061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:46.591539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccd069-857e-4888-86a9-6b1b9a03e061 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:46.597887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 185/356] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=19009ab53c574545b66af5787e770785&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:46.599097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369d01b-eb8b-41c0-b9f2-c2f716539154 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.599775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369d01b-eb8b-41c0-b9f2-c2f716539154 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.599775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369d01b-eb8b-41c0-b9f2-c2f716539154 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.601104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 186/357] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:46.601762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369d01b-eb8b-41c0-b9f2-c2f716539154 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.618710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea684a0-2df4-41e9-b4f5-884edc2bc510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:46.619881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea684a0-2df4-41e9-b4f5-884edc2bc510 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.620172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea684a0-2df4-41e9-b4f5-884edc2bc510 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.620451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea684a0-2df4-41e9-b4f5-884edc2bc510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.732631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea684a0-2df4-41e9-b4f5-884edc2bc510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:46.734359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea684a0-2df4-41e9-b4f5-884edc2bc510 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:46.738220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369d01b-eb8b-41c0-b9f2-c2f716539154 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:46.759699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:46.786745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:46.787531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.787878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.788221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.789211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=19009ab53c574545b66af5787e770785) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:46.789558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:46.791384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:46.804698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 187/358] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=19009ab53c574545b66af5787e770785&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:46.805808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:46.815672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:46.873160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 172/359] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:46.874970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f395c-aa59-4d74-94e6-dfa98fb32e93 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.876087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f395c-aa59-4d74-94e6-dfa98fb32e93 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.876608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f395c-aa59-4d74-94e6-dfa98fb32e93 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.878791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 173/360] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:46.880528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8c097-ba9d-4b34-a842-881d52cbc040 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:46.881447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8c097-ba9d-4b34-a842-881d52cbc040 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:46.881726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8c097-ba9d-4b34-a842-881d52cbc040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:46.903034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bac8c097-ba9d-4b34-a842-881d52cbc040 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:46.904758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bac8c097-ba9d-4b34-a842-881d52cbc040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:46.961936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b066e9-a496-4681-ada8-a7cd585d2559 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.010120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 174/361] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:47.011347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8c097-ba9d-4b34-a842-881d52cbc040 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.021526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31832d3e-1c1b-4fa4-9c93-151e6c038cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.022210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31832d3e-1c1b-4fa4-9c93-151e6c038cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.022492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31832d3e-1c1b-4fa4-9c93-151e6c038cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.022813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31832d3e-1c1b-4fa4-9c93-151e6c038cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.023725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31832d3e-1c1b-4fa4-9c93-151e6c038cb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:47.023854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31832d3e-1c1b-4fa4-9c93-151e6c038cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.025276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31832d3e-1c1b-4fa4-9c93-151e6c038cb1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.035705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 188/362] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.037040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c6035-cd86-4922-b797-6e02a7dc35be None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.037040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c6035-cd86-4922-b797-6e02a7dc35be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.037040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c6035-cd86-4922-b797-6e02a7dc35be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.056220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dd7c6035-cd86-4922-b797-6e02a7dc35be None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:47.058447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dd7c6035-cd86-4922-b797-6e02a7dc35be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:47.069765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:47.125667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8c097-ba9d-4b34-a842-881d52cbc040 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.134975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:47.149171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 189/363] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:47.150955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c6035-cd86-4922-b797-6e02a7dc35be None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.165783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.168909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.169365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.171377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.173564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.175442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.179035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 175/364] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.183675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.288198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c6035-cd86-4922-b797-6e02a7dc35be None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.298731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458782f-7b66-4e2e-89a4-22747ac8d5a9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.316691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:47.329345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.330253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.330542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.330827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] PATH_INFO: `/v3/users/aa0b32b5e1e241e8b1869bd458a4e2b6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.333296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] RBAC: Authorizing `identity:get_user(user_id=aa0b32b5e1e241e8b1869bd458a4e2b6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.334712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.336455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 190/365] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/users/aa0b32b5e1e241e8b1869bd458a4e2b6 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.337308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.341497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.342974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.343591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.344154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.355654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:47.366142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:47.367104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:47.367910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:47.370729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 176/366] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:47.371845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.408540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:47.444482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da431b73-0a8f-4fde-8f86-ec48c6c0ba1b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.483980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7c060-6d38-428b-9cdd-b042afff1b1b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.487060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.487944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.488221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.488491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.489790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.491449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.498351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 191/367] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.498351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.528774 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.529968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.530575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.531200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.542397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:47.550995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:47.551860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:47.552657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:47.555161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 177/368] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:47.556290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.601517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:47.608936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dce91b-8cd3-40c0-bed6-8fdf4f12c7fb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.639406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:47.644714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.645799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.646427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.646760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.647653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] RBAC: Adding query filter params (role.id=4f981d11e2644f2ea3063f3f139a7bb7, user.id=aa0b32b5e1e241e8b1869bd458a4e2b6, scope.system=all) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:47.648027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.650342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.663933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 192/369] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/role_assignments?user.id=aa0b32b5e1e241e8b1869bd458a4e2b6&scope.system=all&role.id=4f981d11e2644f2ea3063f3f139a7bb7 => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.664691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.675660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaf555-720b-433a-8a36-670af2ee5da1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.725926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.727292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.727775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.728217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.729162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:47.729656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.732033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.745657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 178/370] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.747141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.794932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52b1e1-0787-4422-ad66-c2c813619875 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.841433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.842427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.843242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.843702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.845833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:47.846374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.848050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.861315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 193/371] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.862240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:47.902552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:47.910835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0ee06-3016-4bcc-85a3-5d20b8ebfb4d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:47.956618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:47.957791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:47.958917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:47.959549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:47.960559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:47.961147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:47.962940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:47.974789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 179/372] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:47.980609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:48.028016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:48.112361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7465e58-ddde-4645-af3b-3c41085e531c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:48.150710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f36ef-1edc-41d3-a849-69178f8fe350 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:48.154085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:48.154906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:48.156522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:48.157019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:48.195547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:48.200034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:48.201121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:48.201597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:48.202058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:48.248536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:48.250108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:48.286483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:48.289575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:48.375222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 180/373] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 06:59:48.379532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:48.381309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:48.449551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:48.490028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 194/374] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:48.495228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:48.496694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eedcd5-6a2b-44d7-b815-375921d8d2d6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:48.539011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:48.539810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:48.540652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:48.540933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] PATH_INFO: `/v3/projects/91c604937dd14031814263870ccac629` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:48.542404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] RBAC: Authorizing `identity:get_project(project_id=91c604937dd14031814263870ccac629)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:48.543876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:48.546004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 181/375] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:48 2022] GET /identity/v3/projects/91c604937dd14031814263870ccac629 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:48.554182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:48.674814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52cf94-957b-4e3c-a398-d3a5d9521721 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:48.707231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:48.715991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:48.716884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:48.717195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:48.717469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] PATH_INFO: `/v3/groups/b2d919a3e7e846c4be7cd4c66bcff6be` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:48.723310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29feac9b-154d-4e90-970a-132a160bb83b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:48.725200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] RBAC: Authorizing `identity:get_group(group_id=b2d919a3e7e846c4be7cd4c66bcff6be)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:48.726672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:48.728763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 182/376] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:48 2022] GET /identity/v3/groups/b2d919a3e7e846c4be7cd4c66bcff6be => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:48.734459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:48.826407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:48.827223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:48.827911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:48.828208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:48.833794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:48.835268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:48.837360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 195/377] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:48 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:48.842376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:48.847320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5711c6-e38a-4765-a26e-005e47d38890 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:48.886465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:48.887135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:48.887417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:48.887693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:48.888818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] RBAC: Adding query filter params (group.id=b2d919a3e7e846c4be7cd4c66bcff6be, role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=91c604937dd14031814263870ccac629) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:48.889204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:48.891037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:48.898642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 183/378] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:48 2022] GET /identity/v3/role_assignments?group.id=b2d919a3e7e846c4be7cd4c66bcff6be&scope.project.id=91c604937dd14031814263870ccac629&role.id=aaca343d2e4a43c88674834f08499f15 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:48.904441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:48.997319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014073d5-7e0c-4a96-969a-f54ee02d2344 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:49.036993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a817e-e193-4dc2-ada9-fdf36fa3712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:49.037896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a817e-e193-4dc2-ada9-fdf36fa3712f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.038175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a817e-e193-4dc2-ada9-fdf36fa3712f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.038447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a817e-e193-4dc2-ada9-fdf36fa3712f None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.039634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a817e-e193-4dc2-ada9-fdf36fa3712f None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:49.041121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a817e-e193-4dc2-ada9-fdf36fa3712f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:49.043189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 196/379] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:48 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:49.052500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:49.125312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:49.180881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf154c63-195e-4d78-89eb-612c5eebc029 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:49.225859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50410d8b-9743-4d99-837b-79a767f55e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:49.226656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50410d8b-9743-4d99-837b-79a767f55e00 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.226939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50410d8b-9743-4d99-837b-79a767f55e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.227212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50410d8b-9743-4d99-837b-79a767f55e00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.228050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50410d8b-9743-4d99-837b-79a767f55e00 None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:49.228361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50410d8b-9743-4d99-837b-79a767f55e00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:49.229964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50410d8b-9743-4d99-837b-79a767f55e00 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:49.231758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1cfb0-898b-446f-9bed-36cf4fb4c229 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.232039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1cfb0-898b-446f-9bed-36cf4fb4c229 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.232313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1cfb0-898b-446f-9bed-36cf4fb4c229 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.234219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 197/380] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:49.240283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 184/381] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:49 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:49.241695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e611b8-058d-4ded-ab89-6d4f9d261797 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.241985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e611b8-058d-4ded-ab89-6d4f9d261797 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.242259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e611b8-058d-4ded-ab89-6d4f9d261797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.278737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b7e611b8-058d-4ded-ab89-6d4f9d261797 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:49.279964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b7e611b8-058d-4ded-ab89-6d4f9d261797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:49.381393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f8634-3cde-43cc-8397-a5efbc712782 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.381722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f8634-3cde-43cc-8397-a5efbc712782 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.381956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f8634-3cde-43cc-8397-a5efbc712782 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.383677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 185/382] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:49.385004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 198/383] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:49.397021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd90f-372f-44fb-a53d-3c051236a6d5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.397345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd90f-372f-44fb-a53d-3c051236a6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.397617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd90f-372f-44fb-a53d-3c051236a6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.413893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e611b8-058d-4ded-ab89-6d4f9d261797 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:49.419345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-236cd90f-372f-44fb-a53d-3c051236a6d5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:49.420644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-236cd90f-372f-44fb-a53d-3c051236a6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:49.443011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:49.514667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 186/384] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:49.552747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cd90f-372f-44fb-a53d-3c051236a6d5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:49.588853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e611b8-058d-4ded-ab89-6d4f9d261797 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:49.615230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:49.679186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cd90f-372f-44fb-a53d-3c051236a6d5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:49.685749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:49.686592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.686974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.687252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.721716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:49.729805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:49.730514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:49.731400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:49.732185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:49.732792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:49.732792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:49.732792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:49.732970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 199/385] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:49.738714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:49.756465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:49.775323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:49.776280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:49.777117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:49.781195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 187/386] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:49.785940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:49.893447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:49.927012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7934-f87f-42d3-8fad-57097d3ae97a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:49.961009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1570cde-c16a-4bdc-8b64-ba8df244af5d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:49.975684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:50.009787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.010499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.010786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.011060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.011827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:50.012131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.013919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.015609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.016486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.016922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.017321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.018167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:50.018562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.020033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.031818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 188/387] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:50.033036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 200/388] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:50.037400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:50.042018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:50.145629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6469d1-6cd9-41f9-8648-71c186f91a91 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:50.181131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.182148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.182523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.182888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.184253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.185930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.188019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 189/389] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:50 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:50.193458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:50.212465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b088b-49d4-4a9e-b8f6-6f2f9ae97677 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:50.239278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:50.271692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.272810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.273246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.273683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.275096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.276687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.278978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 201/390] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:50 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:50.284514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:50.301417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28deb-690b-490d-bd9e-46072330256e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:50.305456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:50.338417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.345080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.345080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.345080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.345080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.345080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.345080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 190/391] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:50 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:50.350446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:50.459841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8937da-7e5b-4301-a591-7579c21b2429 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:50.466412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d432bf-a07c-43c5-aae9-85c2ad1df4e8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:50.477578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:50.502899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede410d-8d99-4d8f-a0db-e9a28e0048b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.503972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede410d-8d99-4d8f-a0db-e9a28e0048b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.504356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede410d-8d99-4d8f-a0db-e9a28e0048b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.504722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede410d-8d99-4d8f-a0db-e9a28e0048b5 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785/groups/0efd078227bc4df7abde156da5e614e5/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.507154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede410d-8d99-4d8f-a0db-e9a28e0048b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=19009ab53c574545b66af5787e770785, group_id=0efd078227bc4df7abde156da5e614e5, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.508744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede410d-8d99-4d8f-a0db-e9a28e0048b5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.537760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.538687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.538970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.539242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.546562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.548732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.550569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 191/392] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 06:59:50 2022] PUT /identity/v3/projects/19009ab53c574545b66af5787e770785/groups/0efd078227bc4df7abde156da5e614e5/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 06:59:50.551057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 202/393] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:50 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:50.562775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:50.676142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:50.707416 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104a914-df50-4e8b-be8f-33765dacd124 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:50.745390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc78736-d888-4795-9778-bc65e6691ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:50.746204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc78736-d888-4795-9778-bc65e6691ded None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.746488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc78736-d888-4795-9778-bc65e6691ded None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.746758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc78736-d888-4795-9778-bc65e6691ded None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785/groups/0efd078227bc4df7abde156da5e614e5/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.749166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc78736-d888-4795-9778-bc65e6691ded None admin] RBAC: Authorizing `identity:create_grant(project_id=19009ab53c574545b66af5787e770785, group_id=0efd078227bc4df7abde156da5e614e5, role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:50.750722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc78736-d888-4795-9778-bc65e6691ded None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:50.774753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 203/394] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 06:59:50 2022] PUT /identity/v3/projects/19009ab53c574545b66af5787e770785/groups/0efd078227bc4df7abde156da5e614e5/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:50.843492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3442d-6b7a-4800-9dd3-b21c91a37b1a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.844158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3442d-6b7a-4800-9dd3-b21c91a37b1a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.844621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3442d-6b7a-4800-9dd3-b21c91a37b1a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.846307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 192/395] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:50.852523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700627b-34a2-48fb-b61f-4fbaaf423ec6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:50.852823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700627b-34a2-48fb-b61f-4fbaaf423ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:50.853112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700627b-34a2-48fb-b61f-4fbaaf423ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:50.879334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2700627b-34a2-48fb-b61f-4fbaaf423ec6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:50.880533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2700627b-34a2-48fb-b61f-4fbaaf423ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:50.972694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 204/396] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:51.007582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3442d-6b7a-4800-9dd3-b21c91a37b1a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:51.149757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3442d-6b7a-4800-9dd3-b21c91a37b1a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:51.192627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:51.193556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:51.193837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:51.194106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:51.209925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:51.220219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:51.221329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:51.222166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 193/397] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:51.229857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700627b-34a2-48fb-b61f-4fbaaf423ec6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:51.340704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:51.384527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700627b-34a2-48fb-b61f-4fbaaf423ec6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:51.430669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:51.431361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:51.431762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:51.431934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:51.432651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:51.432957 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:51.434375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:51.449941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 205/398] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:51.455112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:51.508592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:51.600011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cecf9a-5333-4f47-b0dc-a3c0fabd785a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:51.652971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4aa02-4cdd-4f88-8654-76b0ba33edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:51.653871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4aa02-4cdd-4f88-8654-76b0ba33edb2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:51.654146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4aa02-4cdd-4f88-8654-76b0ba33edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:51.654415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4aa02-4cdd-4f88-8654-76b0ba33edb2 None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:51.656729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4aa02-4cdd-4f88-8654-76b0ba33edb2 None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:51.658223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4aa02-4cdd-4f88-8654-76b0ba33edb2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:51.660241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 194/399] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:51 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:51.665474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:51.762824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dd6d7-616c-4eaf-b99d-9b414d4649c9 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:51.763190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dd6d7-616c-4eaf-b99d-9b414d4649c9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:51.763465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dd6d7-616c-4eaf-b99d-9b414d4649c9 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:51.764841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 195/400] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:51.770718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c215-e5fb-42d4-85f4-dd9912f24c1e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:51.771009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c215-e5fb-42d4-85f4-dd9912f24c1e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:51.771292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c215-e5fb-42d4-85f4-dd9912f24c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:51.801596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7c33c215-e5fb-42d4-85f4-dd9912f24c1e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:51.803032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7c33c215-e5fb-42d4-85f4-dd9912f24c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:51.807186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9f927-3e13-4f58-af58-95e7e2de48c7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:51.859440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:51.866238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:51.871914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:51.872376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:51.872792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:51.874709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:51.877264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:51.879906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 206/401] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:51 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:51.886033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:51.916809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 196/402] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:51.954568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33c215-e5fb-42d4-85f4-dd9912f24c1e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:52.006553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:52.066680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69b726-0026-4087-a12e-73d5eb706161 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:52.102168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393272f-5cad-4674-ad48-5771ef1ca101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:52.103344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393272f-5cad-4674-ad48-5771ef1ca101 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.103618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393272f-5cad-4674-ad48-5771ef1ca101 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.103889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393272f-5cad-4674-ad48-5771ef1ca101 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/groups/0efd078227bc4df7abde156da5e614e5/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.105856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393272f-5cad-4674-ad48-5771ef1ca101 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03bdf0b01b44e5abf418124cac9cab, group_id=0efd078227bc4df7abde156da5e614e5, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:52.107404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393272f-5cad-4674-ad48-5771ef1ca101 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:52.126323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 207/403] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 06:59:51 2022] PUT /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/groups/0efd078227bc4df7abde156da5e614e5/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:52.127719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab49ef-d56e-45ea-a71d-0bdd25e28fb4 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.128051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab49ef-d56e-45ea-a71d-0bdd25e28fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.128328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab49ef-d56e-45ea-a71d-0bdd25e28fb4 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.129297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33c215-e5fb-42d4-85f4-dd9912f24c1e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:52.129725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 208/404] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:52.135512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c10a-84a5-4e59-bc47-a4f74dfec3cd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.135791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c10a-84a5-4e59-bc47-a4f74dfec3cd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.136069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c10a-84a5-4e59-bc47-a4f74dfec3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.154077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6759c10a-84a5-4e59-bc47-a4f74dfec3cd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:52.155267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6759c10a-84a5-4e59-bc47-a4f74dfec3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:52.178641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:52.180310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.180971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.181411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.197163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:52.208386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:52.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:52.210351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:52.217399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 197/405] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:52.217954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:52.233399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:52.290523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 209/406] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:52.323147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:52.323340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c10a-84a5-4e59-bc47-a4f74dfec3cd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:52.342216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8192e69-f73f-4b5a-9457-13d6e78278b6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:52.392227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:52.393309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.393824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.394301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.395577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:52.396095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:52.397978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:52.415840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 198/407] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:52.420224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:52.474376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c10a-84a5-4e59-bc47-a4f74dfec3cd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:52.544922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4e0aa-4791-4230-b23c-56721da615fd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:52.586503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:52.587766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.588253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.588724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.590256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:52.591916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:52.593786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:52.594430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 199/408] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:52 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:52.594689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.595002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.595244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.606125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:52.610861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:52.624680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:52.625706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 210/409] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:52.626511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:52.644058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:52.717448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:52.797514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b67a58-2855-4bb1-a9d0-923ddd326cef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:52.815957 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedd33f-ac09-4474-8f83-3bdc5a8d4258 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:52.863846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:52.864734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.865016 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.865324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.866921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:52.868920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:52.877449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 200/410] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:52 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:52.878103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:52.878859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:52.879144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:52.879421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:52.880187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:52.880496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:52.881975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:52.884553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:52.903669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 211/411] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:52.908453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:53.047407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:53.075833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070e98-59ed-4773-8a33-6ce314d10365 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:53.097075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b49b44-a16c-4aeb-beea-8786163d8432 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:53.119010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:53.141544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0582bb-6992-4a97-a0e2-82448862bccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:53.142436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0582bb-6992-4a97-a0e2-82448862bccb None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.142714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0582bb-6992-4a97-a0e2-82448862bccb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.142984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0582bb-6992-4a97-a0e2-82448862bccb None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/groups/0efd078227bc4df7abde156da5e614e5/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.147937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0582bb-6992-4a97-a0e2-82448862bccb None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03bdf0b01b44e5abf418124cac9cab, group_id=0efd078227bc4df7abde156da5e614e5, role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:53.149411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0582bb-6992-4a97-a0e2-82448862bccb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:53.165410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:53.166243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.166495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.166777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] PATH_INFO: `/v3/groups/b2d919a3e7e846c4be7cd4c66bcff6be` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.167977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] RBAC: Authorizing `identity:get_group(group_id=b2d919a3e7e846c4be7cd4c66bcff6be)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:53.169481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:53.172583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 212/412] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:52 2022] GET /identity/v3/groups/b2d919a3e7e846c4be7cd4c66bcff6be => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:53.179060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 201/413] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 06:59:52 2022] PUT /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab/groups/0efd078227bc4df7abde156da5e614e5/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 06:59:53.179830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:53.223863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d7fde-d3f3-4395-b838-35e6780e4a28 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.224183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d7fde-d3f3-4395-b838-35e6780e4a28 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.224427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d7fde-d3f3-4395-b838-35e6780e4a28 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.229072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 202/414] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 06:59:53.235454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1bb12-e87d-4903-8f52-1d66481fac49 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.235768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1bb12-e87d-4903-8f52-1d66481fac49 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.236012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1bb12-e87d-4903-8f52-1d66481fac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.258763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0cb1bb12-e87d-4903-8f52-1d66481fac49 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:53.260508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0cb1bb12-e87d-4903-8f52-1d66481fac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:53.353018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eae58a-a96a-4180-b2ad-4335fe814bcd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:53.393716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 203/415] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:53.403094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:53.404268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.404767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.405293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] PATH_INFO: `/v3/projects/91c604937dd14031814263870ccac629` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.407282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=91c604937dd14031814263870ccac629)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:53.409038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:53.411311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 213/416] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:53 2022] GET /identity/v3/projects/91c604937dd14031814263870ccac629 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:53.417633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1bb12-e87d-4903-8f52-1d66481fac49 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:53.425888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:53.492273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:53.517630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:53.549439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1bb12-e87d-4903-8f52-1d66481fac49 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:53.560598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a6bb8-ab6f-4533-84e0-4df8ce664ca9 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:53.586142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac193f4-f1d3-4241-97a4-f93ece7f8ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:53.587207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac193f4-f1d3-4241-97a4-f93ece7f8ad3 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.587590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac193f4-f1d3-4241-97a4-f93ece7f8ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.587960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac193f4-f1d3-4241-97a4-f93ece7f8ad3 None admin] PATH_INFO: `/v3/projects/91c604937dd14031814263870ccac629/groups/b2d919a3e7e846c4be7cd4c66bcff6be/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.590282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac193f4-f1d3-4241-97a4-f93ece7f8ad3 None admin] RBAC: Authorizing `identity:create_grant(project_id=91c604937dd14031814263870ccac629, group_id=b2d919a3e7e846c4be7cd4c66bcff6be, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:53.591931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac193f4-f1d3-4241-97a4-f93ece7f8ad3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:53.597745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:53.598611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.598898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.599153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.608519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 204/417] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 06:59:53 2022] PUT /identity/v3/projects/91c604937dd14031814263870ccac629/groups/b2d919a3e7e846c4be7cd4c66bcff6be/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 06:59:53.612022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f8e69-5945-4e26-8cca-53f8460c241a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.612567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f8e69-5945-4e26-8cca-53f8460c241a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.613086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f8e69-5945-4e26-8cca-53f8460c241a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.615076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 205/418] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:53.624626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f25f1-3297-43b4-b356-6171eca01fa1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.625816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f25f1-3297-43b4-b356-6171eca01fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.626619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:53.631867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f25f1-3297-43b4-b356-6171eca01fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:53.641748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:53.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:53.643597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:53.647309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 214/419] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:53.656804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:53.669483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-169f25f1-3297-43b4-b356-6171eca01fa1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:53.671053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-169f25f1-3297-43b4-b356-6171eca01fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:53.787400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 206/420] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:53.825157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f25f1-3297-43b4-b356-6171eca01fa1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:53.844630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0581f-92bd-40a8-b034-550850981f09 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:53.852245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:53.883993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:53.884707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:53.884990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:53.885287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:53.886106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:53.886420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:53.887821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:53.897329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:53.898650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 215/421] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:53.904010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:53.955267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f25f1-3297-43b4-b356-6171eca01fa1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:54.013988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:54.016204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.018026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.018511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.042977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:54.060229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:54.061253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:54.062106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:54.065390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 207/422] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:54.071419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:54.088421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b171d6-3c5a-48ef-9c8b-4260a499e9d3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:54.159198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:54.160266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.160756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.224988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:54.270077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:54.271809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af8b5a-1b08-4e00-b251-0168c0c7163a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:54.272350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:54.328657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:54.337999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:54.345190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.345190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.345190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.345190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:54.345190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:54.345190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:54.351335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 208/423] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:54.362070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:54.390389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 216/424] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 06:59:54.399137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:54.486883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:54.543970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2f38-0937-4f0f-a8a3-4af194781ad2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:54.579952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acfa7b-dfce-4aad-be50-aad09d9fd212 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:54.583273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526810e-26ce-4675-a9f9-52ce48f83aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:54.584386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526810e-26ce-4675-a9f9-52ce48f83aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.584902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526810e-26ce-4675-a9f9-52ce48f83aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.585391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526810e-26ce-4675-a9f9-52ce48f83aa8 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.586807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526810e-26ce-4675-a9f9-52ce48f83aa8 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:54.588489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526810e-26ce-4675-a9f9-52ce48f83aa8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:54.595281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 217/425] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:54 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:54.596908 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962372e-3f03-49d3-992a-1e7d2c403e31 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.597427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962372e-3f03-49d3-992a-1e7d2c403e31 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.597918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962372e-3f03-49d3-992a-1e7d2c403e31 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.599649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 218/426] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:54.604374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962372e-3f03-49d3-992a-1e7d2c403e31 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:54.619811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:54.621179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.621679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.622161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.679571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:54.694576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:54.696347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:54.723705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962372e-3f03-49d3-992a-1e7d2c403e31 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:54.761649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7347a8-c60e-4039-8878-183dbdee7968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:54.762503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7347a8-c60e-4039-8878-183dbdee7968 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.762790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7347a8-c60e-4039-8878-183dbdee7968 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.763066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7347a8-c60e-4039-8878-183dbdee7968 None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.764347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7347a8-c60e-4039-8878-183dbdee7968 None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:54.766710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7347a8-c60e-4039-8878-183dbdee7968 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:54.773044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 219/427] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:54 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:54.773044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f660f-13a8-48ab-b539-60b39abea0d5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.773044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f660f-13a8-48ab-b539-60b39abea0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.773044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f660f-13a8-48ab-b539-60b39abea0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.777856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:54.791423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-987f660f-13a8-48ab-b539-60b39abea0d5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:54.792668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-987f660f-13a8-48ab-b539-60b39abea0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:54.807909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 209/428] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:54.809038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:54.877132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 220/429] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:54.877954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f660f-13a8-48ab-b539-60b39abea0d5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:54.904557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:54.928373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb6621-c643-49bf-b03d-9d7d831fd691 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:54.963070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:54.963802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:54.964084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:54.964358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:54.965229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=19009ab53c574545b66af5787e770785) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:54.965537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:54.966966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:54.973348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 210/430] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:54 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=19009ab53c574545b66af5787e770785&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:54.974255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:54.992147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f660f-13a8-48ab-b539-60b39abea0d5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.034222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.035014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.035294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.035564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] PATH_INFO: `/v3/projects/19009ab53c574545b66af5787e770785` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.036865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] RBAC: Authorizing `identity:get_project(project_id=19009ab53c574545b66af5787e770785)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:55.038557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:55.040683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 221/431] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:54 2022] GET /identity/v3/projects/19009ab53c574545b66af5787e770785 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:55.046256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.049735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:55.111668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4e12-9d80-40d1-91b8-ad56f7a93be7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.150555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.151375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.151655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.151928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.164377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:55.173200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:55.174200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:55.175010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 211/432] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:55.180859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.181938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fdcab-ec8f-4a0f-add7-a70469b86d58 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.195739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:55.234022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.234976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.235215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.235552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.236861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:55.238515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:55.240580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 222/433] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:55.246146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.292997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df31b67-7c72-45bb-a12b-3e22a5ad1956 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.323444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:55.332810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.333525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.333812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.334102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.334917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:55.335228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:55.336674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:55.346598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 212/434] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:55.353628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.366446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47938aff-5fba-460c-845b-63b9d96b891f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.415110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee4874-9903-4e3d-9054-1926b7b3d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.415731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4874-9903-4e3d-9054-1926b7b3d6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.416108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4874-9903-4e3d-9054-1926b7b3d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.416493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4874-9903-4e3d-9054-1926b7b3d6e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.417526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee4874-9903-4e3d-9054-1926b7b3d6e6 None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=19009ab53c574545b66af5787e770785) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:55.417945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee4874-9903-4e3d-9054-1926b7b3d6e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:55.419692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee4874-9903-4e3d-9054-1926b7b3d6e6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:55.427616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 223/435] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=19009ab53c574545b66af5787e770785&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:55.444600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:55.486658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ca95-1fbc-4b0f-b59f-3cce074257c3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.527108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.528187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.528666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.529168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.541683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee39e9-32b5-41d1-9175-e56a7ab6a20f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.542028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee39e9-32b5-41d1-9175-e56a7ab6a20f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.542237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee39e9-32b5-41d1-9175-e56a7ab6a20f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.548455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 224/436] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:55.550998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac72d1e-ad79-47fa-a806-a93b0d84335f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.551283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac72d1e-ad79-47fa-a806-a93b0d84335f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.551626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac72d1e-ad79-47fa-a806-a93b0d84335f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.570800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bac72d1e-ad79-47fa-a806-a93b0d84335f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:55.572098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bac72d1e-ad79-47fa-a806-a93b0d84335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:55.612174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:55.616182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:55.655575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 225/437] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 06:59:55.679784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac72d1e-ad79-47fa-a806-a93b0d84335f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.710514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:55.751237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 213/438] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:55.758381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.764328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:55.814628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac72d1e-ad79-47fa-a806-a93b0d84335f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.864364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.866228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.866731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.867195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.879823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:55.880620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2acaf5-5688-4516-8712-5aaebe88ba40 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:55.887554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:55.888452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 06:59:55.889688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 06:59:55.893013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 226/439] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 06:59:55.896909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.919926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:55.921424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:55.921971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:55.922516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:55.924960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:55.927230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:55.932080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 214/440] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:55.935086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:55.986962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.043004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6788a9-03b9-49e7-abea-810aa48270b3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.072669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.078678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ca7d1-be0e-4688-aea1-69e45d67ec0f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.088039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.088730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.089011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.089332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.090271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:56.090379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.091778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:56.107026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 227/441] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:56.115742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.128089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.129674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.130187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.130664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.132675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.136064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:56.138418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 215/442] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:55 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:56.143973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.262829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087c265-1fa9-4fdb-9612-c0c1ab9d0677 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.265989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9448b6-6dca-4e82-afea-47111bd4bdfa None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.276121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.299346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.299994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.300274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.300549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.302887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05e216-69ab-4922-ae93-584ad024cac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.303546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05e216-69ab-4922-ae93-584ad024cac6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.303828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05e216-69ab-4922-ae93-584ad024cac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.304106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05e216-69ab-4922-ae93-584ad024cac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.304931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05e216-69ab-4922-ae93-584ad024cac6 None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=9b4fe27dba204b05b692207de4fcedd4, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:56.305275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05e216-69ab-4922-ae93-584ad024cac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.306819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05e216-69ab-4922-ae93-584ad024cac6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:56.313821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 216/443] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=9b4fe27dba204b05b692207de4fcedd4 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:56.315709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec95cb3-03fc-48eb-b056-f22f1bb8ca29 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.315709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec95cb3-03fc-48eb-b056-f22f1bb8ca29 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.315930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec95cb3-03fc-48eb-b056-f22f1bb8ca29 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.317264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 217/444] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 06:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 06:59:56.323036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7480-a2f0-40bb-b2b4-a2cb5bc8020a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.323292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7480-a2f0-40bb-b2b4-a2cb5bc8020a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.323522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7480-a2f0-40bb-b2b4-a2cb5bc8020a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.342528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-da4f7480-a2f0-40bb-b2b4-a2cb5bc8020a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 06:59:56.343821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-da4f7480-a2f0-40bb-b2b4-a2cb5bc8020a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 06:59:56.363066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.379712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.381480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:56.447408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 218/445] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 06:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 06:59:56.473258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f7480-a2f0-40bb-b2b4-a2cb5bc8020a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.504863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 228/446] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 06:59:56.511218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.601506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f7480-a2f0-40bb-b2b4-a2cb5bc8020a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.603065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.617695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.639215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20d7cd-2ea8-47c0-8f61-a11ea21c67c1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.642722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.643563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.643844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.644113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.658579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 06:59:56.666307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 06:59:56.667220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 06:59:56.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 219/447] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 06:59:56.673581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.682235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.683180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.683461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.683735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] PATH_INFO: `/v3/projects/1e03bdf0b01b44e5abf418124cac9cab` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.685479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] RBAC: Authorizing `identity:get_project(project_id=1e03bdf0b01b44e5abf418124cac9cab)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.686986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:56.689147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 229/448] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/projects/1e03bdf0b01b44e5abf418124cac9cab => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:56.695034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.778332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482989f6-543a-495f-b935-fd9b9c04a86b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.792402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d42b9-31e0-44c6-abf9-1cf6178456f7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.815281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.815957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.816238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.816508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.817289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:56.817585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.818966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:56.827997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 220/449] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:56.828732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.829726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.830010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.830283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] PATH_INFO: `/v3/groups/0efd078227bc4df7abde156da5e614e5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.831474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] RBAC: Authorizing `identity:get_group(group_id=0efd078227bc4df7abde156da5e614e5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.833041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:56.833651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.834700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 230/450] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/groups/0efd078227bc4df7abde156da5e614e5 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:56.840300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:56.859080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.865874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:56.933225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a606-ef1f-428d-a4f0-934ebe389e5e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.946746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0915-7f0e-4802-8ffd-461e276d0be7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:56.967010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.967652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.967931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.968271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.980613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:56.981309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:56.981588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:56.981864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:56.982695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] RBAC: Adding query filter params (group.id=0efd078227bc4df7abde156da5e614e5, role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=1e03bdf0b01b44e5abf418124cac9cab) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:56.983025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:56.985003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:57.005084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 231/451] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/role_assignments?group.id=0efd078227bc4df7abde156da5e614e5&scope.project.id=1e03bdf0b01b44e5abf418124cac9cab&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 06:59:57.045147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:57.046852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:57.074325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:57.155487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 221/452] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 06:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 06:59:57.161067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:57.252618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:57.268651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810770f-79c3-4a5f-afff-4c8828b19bee None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:57.302858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:57.303735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:57.304037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:57.304310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] PATH_INFO: `/v3/projects/91c604937dd14031814263870ccac629` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:57.305533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] RBAC: Authorizing `identity:get_project(project_id=91c604937dd14031814263870ccac629)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:57.306997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:57.308710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 232/453] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 06:59:57 2022] GET /identity/v3/projects/91c604937dd14031814263870ccac629 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:57.314643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:57.415832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56f107-5185-4466-9762-65bda34bd7b7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:57.449364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 06:59:57.468581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf557a3-45f2-4cb5-a45d-9bb98c0566b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:57.469469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf557a3-45f2-4cb5-a45d-9bb98c0566b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:57.469746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf557a3-45f2-4cb5-a45d-9bb98c0566b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:57.470015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf557a3-45f2-4cb5-a45d-9bb98c0566b4 None admin] PATH_INFO: `/v3/groups/b2d919a3e7e846c4be7cd4c66bcff6be` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:57.471224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf557a3-45f2-4cb5-a45d-9bb98c0566b4 None admin] RBAC: Authorizing `identity:get_group(group_id=b2d919a3e7e846c4be7cd4c66bcff6be)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:57.472655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf557a3-45f2-4cb5-a45d-9bb98c0566b4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:57.474296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 222/454] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 06:59:57 2022] GET /identity/v3/groups/b2d919a3e7e846c4be7cd4c66bcff6be => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 06:59:57.480947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 06:59:57.592438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d234-82e0-4848-a591-f1b99277dedd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 06:59:57.630985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f57c5-5147-4b86-8fbd-cfefac1bdb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 06:59:57.631635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f57c5-5147-4b86-8fbd-cfefac1bdb7a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 06:59:57.631920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f57c5-5147-4b86-8fbd-cfefac1bdb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 06:59:57.632194 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f57c5-5147-4b86-8fbd-cfefac1bdb7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 06:59:57.633023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f57c5-5147-4b86-8fbd-cfefac1bdb7a None admin] RBAC: Adding query filter params (group.id=b2d919a3e7e846c4be7cd4c66bcff6be, role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=91c604937dd14031814263870ccac629) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 06:59:57.633366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f57c5-5147-4b86-8fbd-cfefac1bdb7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 06:59:57.634806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f57c5-5147-4b86-8fbd-cfefac1bdb7a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 06:59:57.641227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 233/455] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 06:59:57 2022] GET /identity/v3/role_assignments?group.id=b2d919a3e7e846c4be7cd4c66bcff6be&scope.project.id=91c604937dd14031814263870ccac629&role.id=aaca343d2e4a43c88674834f08499f15 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:00.011473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef16292-80ce-4a52-afbc-e4504f2bdf64 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.014041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef16292-80ce-4a52-afbc-e4504f2bdf64 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.014041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef16292-80ce-4a52-afbc-e4504f2bdf64 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.014041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 223/456] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:00.019712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494618-7228-4c24-aae2-a60e0c15dfdb None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.020171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494618-7228-4c24-aae2-a60e0c15dfdb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.020272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494618-7228-4c24-aae2-a60e0c15dfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.046272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cf494618-7228-4c24-aae2-a60e0c15dfdb None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:00.047754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cf494618-7228-4c24-aae2-a60e0c15dfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:00.067760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:00.134319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 234/457] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:00.170849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef16292-80ce-4a52-afbc-e4504f2bdf64 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:00.270373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215674f2-5ef9-4008-a836-7fa2b41748a4 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.270548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215674f2-5ef9-4008-a836-7fa2b41748a4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.272920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215674f2-5ef9-4008-a836-7fa2b41748a4 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.274328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 235/458] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:00.279633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ac816-e469-4a6a-9322-03de9fe7975e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.279922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ac816-e469-4a6a-9322-03de9fe7975e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.280204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ac816-e469-4a6a-9322-03de9fe7975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.305740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef16292-80ce-4a52-afbc-e4504f2bdf64 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:00.314055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9a2ac816-e469-4a6a-9322-03de9fe7975e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:00.315383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9a2ac816-e469-4a6a-9322-03de9fe7975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:00.341665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:00.342466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.342747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.343021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.358862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:00.360323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:00.379757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:00.380669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:00.381699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:00.381699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:00.381699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:00.381699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:00.381699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 224/459] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:00.389548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:00.449441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:00.464827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 236/460] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:00.467302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0bd8a-ae66-4d90-8090-94cded1b17be None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.467847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0bd8a-ae66-4d90-8090-94cded1b17be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.468374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0bd8a-ae66-4d90-8090-94cded1b17be None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.470148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 237/461] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:00.485027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd9a96-18b1-4865-b2d2-4599f9f0a09c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.485621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd9a96-18b1-4865-b2d2-4599f9f0a09c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.486117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd9a96-18b1-4865-b2d2-4599f9f0a09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.507269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-92dd9a96-18b1-4865-b2d2-4599f9f0a09c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:00.508787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-92dd9a96-18b1-4865-b2d2-4599f9f0a09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:00.522650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a76f-d185-4723-a96b-4b51bc30eb86 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:00.567985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:00.568905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.569215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.569488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.570633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:00.570941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:00.572813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:00.585163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 238/462] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:00.586352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd9a96-18b1-4865-b2d2-4599f9f0a09c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:00.588254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 225/463] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:00.596843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:00.608436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:00.728054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95492e64-9bf8-4cf8-8278-f1fa8b8d1eb0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:00.730668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd9a96-18b1-4865-b2d2-4599f9f0a09c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:00.731269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:00.770897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:00.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.772567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.772970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:00.773324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.773739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.774239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.774704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.775856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:00.777958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:00.790747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:00.792692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:00.809831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 226/464] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:00:00 2022] POST /identity/v3/users => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:00.811956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:00.815026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:00.815901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:00.816748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:00.816748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:00.816748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:00.816748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:00.816748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 239/465] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:00.822570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:00.936606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:00.939817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcedfb1-8e77-485a-bf99-b27b2be662b8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:00.979321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6693f-64a3-4cbc-a730-02d5a7d034d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:00.980594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6693f-64a3-4cbc-a730-02d5a7d034d5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:00.981160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6693f-64a3-4cbc-a730-02d5a7d034d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:00.981629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6693f-64a3-4cbc-a730-02d5a7d034d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:00.996651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6693f-64a3-4cbc-a730-02d5a7d034d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:00.999755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6693f-64a3-4cbc-a730-02d5a7d034d5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.012419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0c6b8-2571-49de-b35b-36cc275b9c90 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3ea6693f-64a3-4cbc-a730-02d5a7d034d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:01.020892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:01.022165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:01.023086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:01.023086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:01.023086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:01.023086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:01.024926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 227/466] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:01.026137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac48e-405e-4d60-9dce-7ca5437830e2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.026421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac48e-405e-4d60-9dce-7ca5437830e2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.026694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac48e-405e-4d60-9dce-7ca5437830e2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.028105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 228/467] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:01.033405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ac48e-405e-4d60-9dce-7ca5437830e2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:01.052455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:01.065270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d58ed-369b-4cd2-8ab4-45512edf670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:01.066185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d58ed-369b-4cd2-8ab4-45512edf670a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.066539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d58ed-369b-4cd2-8ab4-45512edf670a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.066885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d58ed-369b-4cd2-8ab4-45512edf670a None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.067717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d58ed-369b-4cd2-8ab4-45512edf670a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:01.068118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d58ed-369b-4cd2-8ab4-45512edf670a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:01.074703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d58ed-369b-4cd2-8ab4-45512edf670a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.095051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 240/468] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:01.096524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24a14a-44c0-4bd2-8536-2220dca04474 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.096883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24a14a-44c0-4bd2-8536-2220dca04474 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.097252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24a14a-44c0-4bd2-8536-2220dca04474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.120506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bc24a14a-44c0-4bd2-8536-2220dca04474 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:01.120506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bc24a14a-44c0-4bd2-8536-2220dca04474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:01.182595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ac48e-405e-4d60-9dce-7ca5437830e2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:01.205757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 241/469] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:01.206595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24a14a-44c0-4bd2-8536-2220dca04474 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:01.226015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:01.226917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.227335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.227772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.228669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:01.229195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:01.230812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.249076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 229/470] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:01.250709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:01.268697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:01.335278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24a14a-44c0-4bd2-8536-2220dca04474 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:01.338044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:01.370332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc605d9-3ac3-4c6c-8790-e30d9dfe1d12 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:01.382854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:01.383556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.383839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.384132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.384943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:01.386681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.413305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:01.414186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 242/471] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:00:01 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:01.415046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.415165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:01.415704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.416277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.434104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:01.435843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:01.456011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:01.456993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:01.458377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:01.458377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:01.458377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:01.458377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:01.458377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 230/472] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:01.463570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:01.570783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2ac1d-6121-4f76-bc43-878e6cf5a549 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:01.581585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:01.616782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:01.617421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.617741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.618019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.619253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:01.621073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.649005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 243/473] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:00:01 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:01.664290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a55cb-ebfe-4058-9618-a4900a49d03e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:01.695154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:01.710914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0efd9-997a-4fe1-b377-a55dc5e99b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:01.711618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0efd9-997a-4fe1-b377-a55dc5e99b1f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.711896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0efd9-997a-4fe1-b377-a55dc5e99b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.712312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0efd9-997a-4fe1-b377-a55dc5e99b1f None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.713546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0efd9-997a-4fe1-b377-a55dc5e99b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:01.713856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0efd9-997a-4fe1-b377-a55dc5e99b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:01.715311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0efd9-997a-4fe1-b377-a55dc5e99b1f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.730163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 231/474] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:01.735719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:01.784238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953826f-692f-42ff-b104-2ece23f70ff8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.784545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953826f-692f-42ff-b104-2ece23f70ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.784791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953826f-692f-42ff-b104-2ece23f70ff8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.786584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 232/475] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:01.792609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503b30-952b-49b4-9c5b-703efe1e5278 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.792919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503b30-952b-49b4-9c5b-703efe1e5278 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.793174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503b30-952b-49b4-9c5b-703efe1e5278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.813809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-55503b30-952b-49b4-9c5b-703efe1e5278 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:01.815717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-55503b30-952b-49b4-9c5b-703efe1e5278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:01.864255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c441f-889e-4ddd-bafa-431988299cbf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:01.901721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:01.902326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:01.902606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:01.902878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:01.903592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:01.905026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:01.905148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 233/476] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:01.926155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55503b30-952b-49b4-9c5b-703efe1e5278 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:01.929568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 244/477] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:00:01 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:02.036626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55503b30-952b-49b4-9c5b-703efe1e5278 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:02.053197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:02.075875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:02.076560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:02.076845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:02.077154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:02.078449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:02.080002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:02.088937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.sql.core [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 07:00:02.088937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 07:00:02.088937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [parameters: {'id': 'df44a652be9141a98db12129dc6f9bf5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 07:00:02.088937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80472) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 12 07:00:02.107046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 07:00:02.107980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application yield resource Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise self.value Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application session.commit() Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 07:00:02.109034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.flush() Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application compat.raise_( Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 07:00:02.109948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 07:00:02.110876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 07:00:02.113517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [parameters: {'id': 'df44a652be9141a98db12129dc6f9bf5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:02.120213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 07:00:02.122754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:02.128991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 234/478] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 07:00:01 2022] POST /identity/v3/roles => generated 144 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:02.129284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:02.143706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:02.238326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6014-58eb-4681-9779-26b92bd7f058 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:02.274017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1894d7b-a47e-42fe-b68e-f827ac3d6bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:02.274846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1894d7b-a47e-42fe-b68e-f827ac3d6bcb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:02.275129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1894d7b-a47e-42fe-b68e-f827ac3d6bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:02.275408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1894d7b-a47e-42fe-b68e-f827ac3d6bcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:02.286621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1894d7b-a47e-42fe-b68e-f827ac3d6bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a1894d7b-a47e-42fe-b68e-f827ac3d6bcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:02.294890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:02.297013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 07:00:02.299191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:02.301615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 245/479] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:02.305088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:02.410878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d161a6-d4b2-4fa1-ad0b-9e337b4aa89f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:02.445247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b88192-efeb-4806-835a-55c25cc5e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:02.445931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b88192-efeb-4806-835a-55c25cc5e74d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:02.446217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b88192-efeb-4806-835a-55c25cc5e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:02.446493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b88192-efeb-4806-835a-55c25cc5e74d None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:02.447254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b88192-efeb-4806-835a-55c25cc5e74d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:02.447562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b88192-efeb-4806-835a-55c25cc5e74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:02.449385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b88192-efeb-4806-835a-55c25cc5e74d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:02.462331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 235/480] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:03.360937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d0fe2-9504-41e4-89e1-410b8f5a2726 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:03.361638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d0fe2-9504-41e4-89e1-410b8f5a2726 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:03.361638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d0fe2-9504-41e4-89e1-410b8f5a2726 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:03.363080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 246/481] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:03.368341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e3f3a-8348-49ba-ad16-99719a9d93d8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:03.368826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e3f3a-8348-49ba-ad16-99719a9d93d8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:03.368977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e3f3a-8348-49ba-ad16-99719a9d93d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:03.386330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-579e3f3a-8348-49ba-ad16-99719a9d93d8 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:03.387511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-579e3f3a-8348-49ba-ad16-99719a9d93d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:03.422508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:03.464816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 236/482] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:03.488447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d0fe2-9504-41e4-89e1-410b8f5a2726 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:03.587217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:03.610893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d0fe2-9504-41e4-89e1-410b8f5a2726 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:03.645805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:03.646579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:03.646864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:03.647138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:03.657546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:03.665571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:03.666512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:03.667803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 247/483] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:03.673525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e3f3a-8348-49ba-ad16-99719a9d93d8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:03.784752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e3f3a-8348-49ba-ad16-99719a9d93d8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:03.818353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:03.819035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:03.819355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:03.819638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:03.820431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:03.820741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:03.822166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:03.835298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 237/484] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:03.840835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:03.967624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7e5e1-f075-4eec-8399-0716161f52d2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.004585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42517531-59e4-425b-8b42-394f316996b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.005442 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42517531-59e4-425b-8b42-394f316996b0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.005726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42517531-59e4-425b-8b42-394f316996b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.006001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42517531-59e4-425b-8b42-394f316996b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.012974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:04.017464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42517531-59e4-425b-8b42-394f316996b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:04.018936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42517531-59e4-425b-8b42-394f316996b0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-42517531-59e4-425b-8b42-394f316996b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:04.036775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:04.037714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:04.039167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:04.039167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:04.039167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:04.039167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:04.039167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 248/485] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:04.043745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.084207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:04.154622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bc67-9331-48c9-91ff-913bda3caae7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.185103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c2215-1d0d-43a1-9a0e-2bc24d04bc52 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.185103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c2215-1d0d-43a1-9a0e-2bc24d04bc52 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.185103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c2215-1d0d-43a1-9a0e-2bc24d04bc52 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.187024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 249/486] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:04.189459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.190138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.190419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.190740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.191476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:04.191782 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:04.193156 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3037ab-cc9d-4509-b8b3-d4db8e2b0e68 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.193280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:04.193753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3037ab-cc9d-4509-b8b3-d4db8e2b0e68 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.194224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3037ab-cc9d-4509-b8b3-d4db8e2b0e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.212298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 238/487] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:04.215560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6c3037ab-cc9d-4509-b8b3-d4db8e2b0e68 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:04.217410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6c3037ab-cc9d-4509-b8b3-d4db8e2b0e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:04.217831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.328447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:04.338009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7f16e-ba6f-4fd9-9513-c4d8e6be4014 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.358421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 250/488] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:04.376113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.376921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.377238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.377514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.385447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67f47-b5dc-48e3-a987-ddb857dbd00a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.385909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:04.386411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67f47-b5dc-48e3-a987-ddb857dbd00a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.386914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67f47-b5dc-48e3-a987-ddb857dbd00a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.387233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:04.392567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 251/489] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:04.394152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caeaac-9257-4549-8d4a-979f3e268df1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.394663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caeaac-9257-4549-8d4a-979f3e268df1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.395157 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caeaac-9257-4549-8d4a-979f3e268df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:04.403533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:04.404482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:04.405898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:04.409166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 239/490] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:04.410038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.424100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-91caeaac-9257-4549-8d4a-979f3e268df1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:04.425691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-91caeaac-9257-4549-8d4a-979f3e268df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:04.503046 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 252/491] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:04.503815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91caeaac-9257-4549-8d4a-979f3e268df1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.542775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74eb89-caf4-4be9-8895-a29f6ccf10cd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.565364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:04.581313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.582510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.582992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.583460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.595256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:04.603937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:04.604842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:04.605689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:04.613877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 240/492] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:04.615089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.622614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91caeaac-9257-4549-8d4a-979f3e268df1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.663391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:04.670060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.671019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.671360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.671787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.672747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:04.673489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:04.674265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:04.720390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 253/493] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:04.721276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.738612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29356858-87fe-4e09-858a-d35b30746f40 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.772304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763390c7-b4a2-451f-b4de-96f5f28f2b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.773027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763390c7-b4a2-451f-b4de-96f5f28f2b3d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.773335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763390c7-b4a2-451f-b4de-96f5f28f2b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.773607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763390c7-b4a2-451f-b4de-96f5f28f2b3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.784763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763390c7-b4a2-451f-b4de-96f5f28f2b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-763390c7-b4a2-451f-b4de-96f5f28f2b3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:04.792592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:04.793788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:04.794970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 241/494] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:04.795629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903d25-dde7-42c3-ad12-be78488b8746 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.795629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903d25-dde7-42c3-ad12-be78488b8746 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.795902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903d25-dde7-42c3-ad12-be78488b8746 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.797506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 242/495] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:04.797909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc903d25-dde7-42c3-ad12-be78488b8746 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.837226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744e1ff-0dd1-4561-a1fa-39f90774e7b1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.847776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:04.876229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:04.880775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.881508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.881786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.882062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:04.882818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:04.883128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:04.884531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:04.893191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 254/496] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:04.893987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:04.911453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc903d25-dde7-42c3-ad12-be78488b8746 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:04.949528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:04.950177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:04.950455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:04.950722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.015290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74892657-d08a-42dd-9f23-6a1df0fbe45d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.021580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.023114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.052630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736fc1-cd9c-4bed-afbc-e4eb859e930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.053629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736fc1-cd9c-4bed-afbc-e4eb859e930a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.054045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736fc1-cd9c-4bed-afbc-e4eb859e930a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.054451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736fc1-cd9c-4bed-afbc-e4eb859e930a None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.055465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736fc1-cd9c-4bed-afbc-e4eb859e930a None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:05.056007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736fc1-cd9c-4bed-afbc-e4eb859e930a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.057426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736fc1-cd9c-4bed-afbc-e4eb859e930a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.070439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 255/497] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:05.070439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b3ba6-ede8-4146-bfa7-812701b09172 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.070439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b3ba6-ede8-4146-bfa7-812701b09172 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.070439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b3ba6-ede8-4146-bfa7-812701b09172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.088453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-519b3ba6-ede8-4146-bfa7-812701b09172 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:05.089662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-519b3ba6-ede8-4146-bfa7-812701b09172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:05.095535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.116925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.128374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 243/498] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:05.133432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.159153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 256/499] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:05.160285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b3ba6-ede8-4146-bfa7-812701b09172 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.245756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654838cf-383f-4c41-9d13-c60ae280d79f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.273840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b3ba6-ede8-4146-bfa7-812701b09172 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.283895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.284709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.284989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.285291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.295789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.297444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.314765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.315581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:05.315693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:05.316496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:05.317384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:05.317384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:05.317384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:05.317384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:05.317603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.317603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.317731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 244/500] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:05.317796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.330059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.331545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.345888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:05.350529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:05.351530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:05.352367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:05.352367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:05.352367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:05.352367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:05.352367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 257/501] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:05.352631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.373236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.430810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cf9a1-5221-4b61-8556-b3fa58241fca None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.460322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b55a0-ee37-4adc-b85f-2d3f37b70ca2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.463737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.464542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.464824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.465133 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.476038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.478012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.494118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.494921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.495199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.495470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:05.495719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:05.496533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:05.497436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:05.497436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:05.497436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:05.497436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:05.497436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 245/502] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:05.497795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.506975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:05.514730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:05.515775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:05.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 258/503] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:05.517287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.577964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.596874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.601886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadaf20-d868-4278-b653-b8d94fb7ef0b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.622460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013aa081-9322-47c0-adbb-c2a9b8aaee8e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.635736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee972c2-b685-47a8-b473-d90b56cafbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.636401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee972c2-b685-47a8-b473-d90b56cafbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.636682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee972c2-b685-47a8-b473-d90b56cafbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.636978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee972c2-b685-47a8-b473-d90b56cafbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.637731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee972c2-b685-47a8-b473-d90b56cafbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:05.638038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee972c2-b685-47a8-b473-d90b56cafbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.639433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee972c2-b685-47a8-b473-d90b56cafbf1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.652050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 246/504] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:05.653318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e6b3-b8ce-4456-94fd-251c528708bd None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.653602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e6b3-b8ce-4456-94fd-251c528708bd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.653876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e6b3-b8ce-4456-94fd-251c528708bd None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.655084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 247/505] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:05.655868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e6b3-b8ce-4456-94fd-251c528708bd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.660165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.660930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.661250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.661526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.662261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:05.662566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.663967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.676579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 259/506] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:05.677389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.765039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e6b3-b8ce-4456-94fd-251c528708bd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.779603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162cc75-2d80-4702-9a53-320140bd3585 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.798953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.799636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.799915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.800185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.800893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:05.801219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.802609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.804857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.813885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1d3ae-f44d-462d-a032-1ce23a5aac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.814272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 248/507] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:05.814549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d3ae-f44d-462d-a032-1ce23a5aac0e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.814826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d3ae-f44d-462d-a032-1ce23a5aac0e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.815063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.815157 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d3ae-f44d-462d-a032-1ce23a5aac0e None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.815817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1d3ae-f44d-462d-a032-1ce23a5aac0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:05.816119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1d3ae-f44d-462d-a032-1ce23a5aac0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.817524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1d3ae-f44d-462d-a032-1ce23a5aac0e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.819200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:05.827340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 260/508] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:05.828922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e73ff-55b3-45f5-8524-51f1a4961d9b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.829171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e73ff-55b3-45f5-8524-51f1a4961d9b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.829446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e73ff-55b3-45f5-8524-51f1a4961d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.846410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-238e73ff-55b3-45f5-8524-51f1a4961d9b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:05.847591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-238e73ff-55b3-45f5-8524-51f1a4961d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:05.912743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 261/509] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:05.913792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e73ff-55b3-45f5-8524-51f1a4961d9b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:05.922607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0cff-fecf-4eb1-949c-2581664e53f6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:05.957134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:05.957960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:05.958273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:05.958580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:05.966278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:05.967767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:05.982404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:05.983314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:05.984277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 249/510] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:05.984968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.019061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e73ff-55b3-45f5-8524-51f1a4961d9b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.041617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.045483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.056260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.057102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.057390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.057664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.065779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.067268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:06.084486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:06.085556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 262/511] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:06.086502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.087739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5eb6f-d283-43b7-b775-77954498b0fb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.124061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.125085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.125408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.125711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.132614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.134111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:06.149132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:06.150180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 250/512] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:06.151257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.188604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f167e8-6568-4d34-90d6-37fc0297bcc3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.224155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.224962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.225273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.225549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.235661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.237070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.250891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f0853-34dc-4edc-83a5-f4afb453412b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:06.254702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:06.255544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:06.256406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:06.256406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:06.256406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:06.256406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:06.256406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 263/513] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:06.256664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.268259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.286934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.287725 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.288013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.288293 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.298965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.300172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.300490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:06.317935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:06.318959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:06.319797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:06.319797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:06.319797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:06.319797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:06.319797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 251/514] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:06.320067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.365553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b43ff-e8e4-4516-819d-27390a03c437 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.398384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.399042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.399323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.399590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.400415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:06.400720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.402204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.417808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4dd0b-d8a4-4691-9c1d-60161c28162b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.419371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 264/515] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:06.420179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.451744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.452456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.452759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.453257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.453912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:06.454218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.455700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.492493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 252/516] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:06.493351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.516317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.520106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.522529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e0afb-3784-48f0-9317-39687b95863a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.557146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.557811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.558093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.558368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.559561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:06.559561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.560893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.578642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 265/517] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:06.579436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.591346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06809f14-e024-42d6-a087-973853dc9b33 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.625786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.626367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.626636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.626928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.627649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:06.627941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.629368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.640722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 253/518] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:06.641636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.681630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6b43d-c5f3-49d4-9458-1ddab329a7a5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.717627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.718422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.719084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.719368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.720097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:06.720411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.721830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.730527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.733925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 266/519] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:06.734668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.746447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff4fe3-3dec-405c-9cf3-72cbb0572eaf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.780775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.781598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.781878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.782149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.835119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59953532-cf3a-4c8b-80d8-a4e068ff281a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:06.849204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.850652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.869455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:06.870111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:06.870394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:06.870668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:06.933180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:06.937029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:06.938547 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:06.948142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 254/520] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:06.949334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:06.955577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:07.042172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 267/521] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:07.043014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.055411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cc41b-f467-46ee-b0ce-e7e546e303e6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.089245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.089898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.090181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.090452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.091288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=95c860dcbf3b40bbb7427a417357c68a) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:07.091596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.093078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.098463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 255/522] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:06 2022] GET /identity/v3/role_assignments?user.id=95c860dcbf3b40bbb7427a417357c68a&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:07.099214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.151828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:07.169120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041b863-a11a-4bc1-ac2c-c9f3f3ac94cf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.189018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:07.210608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.211174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5f450-4fcb-489c-a10e-0bed0e5e025c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.211940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.212403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.213003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.223000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.224504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:07.241582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:07.242581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:07.243506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 268/523] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:07.244069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.254231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.254919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.255201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.255469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.256183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.257687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.284418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 256/524] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:00:07 2022] POST /identity/v3/users => generated 247 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:07.285833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.382375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9365cd0-3f24-475c-9a54-0a9ecc148454 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.418225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31598-5a11-4bf4-8696-92deb10ca9be None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.422236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.423646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.424141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.424619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.428654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:07.443367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.445237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.471908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:07.472438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:07.473325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:07.474189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:07.474189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:07.474189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:07.474189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:07.474404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.474404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.474404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.475122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 269/525] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:07.475661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.491763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.493274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:07.514941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:07.515904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:07.516984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:07.516984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:07.516984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:07.516984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:07.516984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 257/526] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:07.518282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.518802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:07.604935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c0b2d-fad3-4bfe-b4a4-7633f762c70c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.629443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bd94a-3c6f-4c8a-b03c-204fb9f0f96a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.645341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.646350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.646823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.647315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.648362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:07.648860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.650498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.669786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 270/527] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:07.670739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.673889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.674664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.674959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.675233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.676001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:07.676300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.677720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.690556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 258/528] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:07.691527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.717455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:07.775994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:07.793358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88161f0-0880-408e-a9a6-ffdb83aa2e14 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.827161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30cdd1-c585-4d2e-bd4b-713e0bcd805a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.831574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.832330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.832680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.833011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.833855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:07.834224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.835698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.848871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 271/529] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:07.850447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:07.872247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:07.873247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:07.873706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:07.874867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:07.961408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f106d-3057-4355-97e2-bf07c84b01b6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:07.969929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:07.971335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:07.990033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:08.009290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.010124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.010407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.010679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.019743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.021249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:08.037229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:08.038134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:08.039005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 272/530] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:08.039936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.049106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:08.106183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 259/531] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:08.106183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.168716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6c8b2-cd21-4492-b31b-00b6ea536b9d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.206725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.207696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.208091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.208486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.215653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.217241 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.218894 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343522c7-d02e-42a3-a56e-6ed1ef5da796 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:08.232525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:08.233655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:08.234870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:08.236700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 273/532] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:08.238008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.244321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:08.268213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.269396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.269822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.270634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.271882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:08.272337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.274840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.299389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 260/533] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:08.300925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.310640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:08.349470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f77ff-73da-4b33-95fc-fd7385bc3b1d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.395044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.396086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.397003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.397486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.411723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.414041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.422526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55aa5-c8a9-4ca0-8bf7-8d22ac89da32 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:08.435196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:08.436238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:08.437380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:08.437380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:08.437380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:08.437380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:08.442788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 274/534] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:08.442788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.476753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.477466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.477753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.478030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.479165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:08.479477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.481152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.496064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 261/535] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:08.498484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.544670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:08.582589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b19275-90c2-4fbf-b444-ffab36aec67b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.606995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:08.615186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4ef6f-45cc-4b39-ad2a-301c97022c90 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.626652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605619-351d-4636-a454-2238b3948365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.627556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605619-351d-4636-a454-2238b3948365 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.628042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605619-351d-4636-a454-2238b3948365 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.628510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605619-351d-4636-a454-2238b3948365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.629552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa605619-351d-4636-a454-2238b3948365 None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=366a7e7596d14b32a6ce62790a2e3dd5) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:08.630037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa605619-351d-4636-a454-2238b3948365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.631638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa605619-351d-4636-a454-2238b3948365 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.637531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 275/536] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/role_assignments?user.id=366a7e7596d14b32a6ce62790a2e3dd5&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:08.638487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605619-351d-4636-a454-2238b3948365 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.658351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.659034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.659579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.659863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.660605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:08.660916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.662864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.679490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 262/537] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:08.686014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:08.765377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605619-351d-4636-a454-2238b3948365 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.804190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.805209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.805667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.806114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.807149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=b4292042e79c45438b5964710359cd5e) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:08.807667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.809361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.816897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 276/538] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/role_assignments?user.id=b4292042e79c45438b5964710359cd5e&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:08.875803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:08.878886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7868-76b1-4dcc-9ee4-e6c88ced25a7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:08.917907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:08.918742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:08.919004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:08.919276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:08.927003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:08.928488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:08.943822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:08.944732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:08.945633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 263/539] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:08.952823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:09.036759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:09.103712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccc79-0eb4-4a49-ae2b-c2b2988f2f68 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:09.148121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83536cc9-6b2b-4a55-be26-6320dd894c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:09.148791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536cc9-6b2b-4a55-be26-6320dd894c62 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:09.149086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536cc9-6b2b-4a55-be26-6320dd894c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:09.149368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536cc9-6b2b-4a55-be26-6320dd894c62 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:09.150165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83536cc9-6b2b-4a55-be26-6320dd894c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:09.150475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83536cc9-6b2b-4a55-be26-6320dd894c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:09.151992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83536cc9-6b2b-4a55-be26-6320dd894c62 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:09.165413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 277/540] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:09.171438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:09.320103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113a94c-c240-4505-b2f1-95f9887e2f21 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:09.361784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec4e62-6291-461e-a25e-3f404352912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:09.362448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4e62-6291-461e-a25e-3f404352912d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:09.362783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4e62-6291-461e-a25e-3f404352912d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:09.363050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4e62-6291-461e-a25e-3f404352912d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:09.363935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4e62-6291-461e-a25e-3f404352912d None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=754c137a554941d99f7ef9c380a55ca0) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:09.364244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4e62-6291-461e-a25e-3f404352912d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:09.365712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4e62-6291-461e-a25e-3f404352912d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:09.371633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 264/541] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:09 2022] GET /identity/v3/role_assignments?user.id=754c137a554941d99f7ef9c380a55ca0&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:09.415402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202f238-45c1-47a1-9aa1-a9b09353885f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:09.415718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202f238-45c1-47a1-9aa1-a9b09353885f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:09.415959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202f238-45c1-47a1-9aa1-a9b09353885f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:09.417726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 278/542] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:09.423868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1501-1656-405e-96d7-947c7392e837 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:09.424224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1501-1656-405e-96d7-947c7392e837 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:09.424431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1501-1656-405e-96d7-947c7392e837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:09.443345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b82d1501-1656-405e-96d7-947c7392e837 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:09.444583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b82d1501-1656-405e-96d7-947c7392e837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:09.465642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:09.543577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 265/543] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:09.578976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202f238-45c1-47a1-9aa1-a9b09353885f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:09.652349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1642-d14e-4c63-8836-0674f65064ca None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:09.652678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1642-d14e-4c63-8836-0674f65064ca None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:09.652911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1642-d14e-4c63-8836-0674f65064ca None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:09.655289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 266/544] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:09.662094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feea76-4044-4128-95f7-66c1879c2df6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:09.662384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feea76-4044-4128-95f7-66c1879c2df6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:09.663537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feea76-4044-4128-95f7-66c1879c2df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:09.689553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-19feea76-4044-4128-95f7-66c1879c2df6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:09.690754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19feea76-4044-4128-95f7-66c1879c2df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:09.710693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202f238-45c1-47a1-9aa1-a9b09353885f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:09.722891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:09.779242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:09.780047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:09.780319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:09.780584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:09.799155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:09.810180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:09.811069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:09.811889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:09.813084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 279/545] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:09.817683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:09.820561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 267/546] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 07:00:09.873969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feea76-4044-4128-95f7-66c1879c2df6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:09.980213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba2d34-369c-4b81-80b9-598f8f6b1f27 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.033449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:10.049458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feea76-4044-4128-95f7-66c1879c2df6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.058310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.059191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.059627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.060042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:10.060958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:10.061439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:10.063934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:10.083676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 280/547] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:10.085896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:10.120587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.121790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.122451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.123693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:10.130501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:10.151862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:10.161005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:10.162021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:10.162844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:10.168956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 268/548] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:10.175662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:10.257873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe35e2-0486-4bde-aae8-3c5a7c81aa51 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.323170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.324701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.325211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.325973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:10.345298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:10.348170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:10.368163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a150e6-677b-4f00-833c-c2e32ec59433 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:10.381934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:10.383245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:10.384418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:10.384418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:10.384418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:10.384418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:10.389676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 281/549] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:10.392404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:10.438206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.439311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.439809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.440321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:10.441288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:10.441795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:10.443391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:10.463039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 269/550] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:10.468880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:10.472808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:10.561367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:10.597646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c942c-62ea-47c6-a744-d8067278e650 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.601011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fffc7-1844-4bce-b597-0255e14059dc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.643278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.644191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.644480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.644755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:10.657518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:10.659591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:10.674763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.675436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.675719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.675991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:10.676731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:10.677031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:10.678437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:10.689188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:10.690177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:10.691169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:10.691169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:10.691169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:10.691169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:10.693391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 270/551] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:10.699332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:10.714199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 282/552] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:10.720065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:10.817422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:10.902615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fca63-a235-4f10-be82-48a328ba7934 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.915322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61792d60-cc72-411f-ae8d-0838880f6ad2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:10.968973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.969529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.969813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.970089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:10.970829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:10.971136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:10.972530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:10.993335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:10.994265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:10.994547 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:10.994820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.004007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 271/553] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:11.004965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.006454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:11.007939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:11.018400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:11.035473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:11.036433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:11.037497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:11.041091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 283/554] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:00:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:11.042548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.187386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f2b7e-2b44-43e3-b62a-0c394b4d19c3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.219482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.226071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15c94c-2eae-4ef0-b18a-b76603ea4854 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.240204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b883b0-18a2-469b-9274-9512fbe741b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.241024 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b883b0-18a2-469b-9274-9512fbe741b5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.241338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b883b0-18a2-469b-9274-9512fbe741b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.241611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b883b0-18a2-469b-9274-9512fbe741b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.249421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b883b0-18a2-469b-9274-9512fbe741b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:11.251517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b883b0-18a2-469b-9274-9512fbe741b5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:11.262302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.263004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.263558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.264405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:11.264712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:11.266570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f8b883b0-18a2-469b-9274-9512fbe741b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:11.268607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:11.269534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:11.271004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 272/555] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:11.271605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735b066-cbb9-44e8-96c2-4021e69f2c34 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.271605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735b066-cbb9-44e8-96c2-4021e69f2c34 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.272227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735b066-cbb9-44e8-96c2-4021e69f2c34 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.273398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 273/556] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:11.274537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c26a1f-d076-4d1a-bb3d-3de95f95d2d0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.274822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c26a1f-d076-4d1a-bb3d-3de95f95d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.275097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c26a1f-d076-4d1a-bb3d-3de95f95d2d0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.277180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 274/557] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:11.279296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908afe7a-64cd-4296-abd1-3df3a687c9b6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.279787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908afe7a-64cd-4296-abd1-3df3a687c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.280369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908afe7a-64cd-4296-abd1-3df3a687c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.290803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 284/558] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:11.292665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.300185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-908afe7a-64cd-4296-abd1-3df3a687c9b6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:11.301437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-908afe7a-64cd-4296-abd1-3df3a687c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:11.362633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.378130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 275/559] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 07:00:11.379590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d98a9-bb1d-4cb6-9b62-636aa999a229 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.379878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d98a9-bb1d-4cb6-9b62-636aa999a229 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.380152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d98a9-bb1d-4cb6-9b62-636aa999a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.397001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7d8d98a9-bb1d-4cb6-9b62-636aa999a229 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:11.398448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d8d98a9-bb1d-4cb6-9b62-636aa999a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:11.416029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c719-8065-44aa-adf0-7a11f9214044 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.452331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.452969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.453294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.453795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.454402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:11.454708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:11.456132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:11.481128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 276/560] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:11.481128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d98a9-bb1d-4cb6-9b62-636aa999a229 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.481433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 285/561] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:11.481433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.510726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.582360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d98a9-bb1d-4cb6-9b62-636aa999a229 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.595587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73148a-7750-4d36-8be8-85caddd52096 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.613425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.621843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.622762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.623047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.623319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.632383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.633222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.633502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.633773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.635030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:11.636508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:11.646047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:11.654052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:11.655176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:11.656594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:11.657440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:11.658258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:11.659221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:11.659221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:11.659221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:11.659221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:11.659221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 277/562] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:11.659221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.660850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 286/563] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:11.661795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.758253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.770452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d47032-c819-43ab-94d9-b8a142ae826a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.774232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6abb-217c-491a-a543-81aa033c731a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.804028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4f79c-76fe-416b-bb1f-2a799843fb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.804687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4f79c-76fe-416b-bb1f-2a799843fb81 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.804970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4f79c-76fe-416b-bb1f-2a799843fb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.805279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4f79c-76fe-416b-bb1f-2a799843fb81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.807480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.808385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.808668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.808939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.819339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:11.827438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:11.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 287/564] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:11.829585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.863629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.877639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4f79c-76fe-416b-bb1f-2a799843fb81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:11.879105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4f79c-76fe-416b-bb1f-2a799843fb81 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:11.928654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d97177-e84f-4cff-a48e-cd2e4be039ba None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:11.962753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:11.963414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.963692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.964681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:11.964986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:11.966393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:11.966824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:11.981918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 288/565] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:11.981918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:11.982273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 278/566] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:11.982273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb53ac-2208-48e8-a55f-95aeebf5992f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:11.982273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb53ac-2208-48e8-a55f-95aeebf5992f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:11.982273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb53ac-2208-48e8-a55f-95aeebf5992f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:11.987439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 279/567] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:11.987439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb53ac-2208-48e8-a55f-95aeebf5992f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.078205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.084299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbe3c-2716-467d-b22e-7271956c449f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.090445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb53ac-2208-48e8-a55f-95aeebf5992f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.116979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.117672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.117952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.118225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.118956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:12.119257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.120640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.125076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.125767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.126048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.126330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.127064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:12.127366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.128708 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.133471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 289/568] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:12.134359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.136952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 280/569] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:12.137768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.187401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.237379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909f173-337f-4904-a3a0-4b802ffb68b6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.246066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2f57a-7bca-4a28-bc1a-32712e0b48d2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.272291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077738d-3656-47f8-8765-aa2619187c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.273085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077738d-3656-47f8-8765-aa2619187c62 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.273379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077738d-3656-47f8-8765-aa2619187c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.273647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077738d-3656-47f8-8765-aa2619187c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.281754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.282537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.282818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.283092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.284127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077738d-3656-47f8-8765-aa2619187c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.285622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077738d-3656-47f8-8765-aa2619187c62 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.289775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.291205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d077738d-3656-47f8-8765-aa2619187c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:12.302677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:12.303612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:12.304495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:12.304495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:12.304495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:12.304495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:12.304495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 281/570] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:12.305200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba7b48-55aa-4cc3-bcf5-febf3805d14a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.305296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba7b48-55aa-4cc3-bcf5-febf3805d14a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.305603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba7b48-55aa-4cc3-bcf5-febf3805d14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:12.305804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:12.306663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 290/571] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:12.308511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.327853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1aba7b48-55aa-4cc3-bcf5-febf3805d14a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:12.328036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.329865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1aba7b48-55aa-4cc3-bcf5-febf3805d14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:12.395805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 282/572] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:12.396875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba7b48-55aa-4cc3-bcf5-febf3805d14a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.412053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95919d16-9f5a-41b3-a586-b9bfa02d6e63 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.426924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.446405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.447209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.447490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.447767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.458274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.459745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:12.477755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:12.478689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:12.479585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:12.479585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:12.479585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:12.479585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:12.479585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 291/573] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:12.479893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.499928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba7b48-55aa-4cc3-bcf5-febf3805d14a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.533732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.534537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.534816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.535093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.545252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.546776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.546930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:12.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:12.565329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:12.566234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:12.566234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:12.566234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:12.566234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:12.566234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 283/574] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:12.567255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.580106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463b9a0-c027-4190-9ae4-525cd82bb8c8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.624730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.625564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.625849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.626119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.626872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:12.627172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.629174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.644355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 292/575] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:12.645217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.659326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.675932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64edd366-4512-45e2-9f1b-72ee6f71ce66 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.710216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.710910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.711191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.711463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.712255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:12.712563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.714023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.728906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 284/576] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:12.729768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.753626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52da3e-2d48-42b9-a500-a9f22b835143 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.783341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.790117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.790948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.791233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.791508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.801927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:12.809740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:12.810850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:12.811690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 293/577] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:12.812491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.838033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7bc65-ed25-4b70-9b00-856410c30648 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.872271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.872937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.873242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.873513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.874229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:12.874663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.876164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.887848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 285/578] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:12.888669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:12.893132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:12.923227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8a0b-a1e2-4daf-9484-16dae8e678a2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:12.959800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:12.960606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:12.960879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:12.961177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:12.961886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:12.962186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:12.963570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:12.976351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 294/579] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:12.977177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.017715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f7ab-faac-4a7b-8d63-08797d690685 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.036096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:13.056118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.058104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.058387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.059617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.071859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.073592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:13.089170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:13.090185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:13.091271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 286/580] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:13.091873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.095915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c148ff4-0fe5-4c5b-90ab-dc742dd42bde None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.136263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.137188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.137455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.137998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/95c860dcbf3b40bbb7427a417357c68a/roles/59c2d2f6ca754e289f3766b15292756e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.152683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=95c860dcbf3b40bbb7427a417357c68a, role_id=59c2d2f6ca754e289f3766b15292756e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.154177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.164511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:13.168369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 295/581] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:12 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/95c860dcbf3b40bbb7427a417357c68a/roles/59c2d2f6ca754e289f3766b15292756e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:13.169216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.223239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6603e8e-e5f8-4f2a-84c1-aadb32f632b3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.264900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.265807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.266085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.266350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.267104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:13.267404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.269092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.282920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 287/582] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:13.282920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.295036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:13.300118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50579255-4564-425d-a1da-b6e0c3629e45 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.351596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.352332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.352613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.352889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.361162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.362647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:13.379029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:13.380496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 296/583] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:13.381508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.414743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608043d-5643-4bf2-b884-324f33e00bde None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.445436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:13.455868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.456713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.457005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.457308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.466655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.468274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:13.486173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:13.487227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:13.488172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 288/584] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:13.489276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.505740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a3fe7-4e74-4d04-8534-623a830660fb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.543190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.543866 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.544127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.544418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.545359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:13.545664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.547519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.567562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 297/585] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:13.567562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.590540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:13.605441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3f159-d3c5-44a6-9c19-981b88395141 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.647123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.648295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.648589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.648853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.662926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.664412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:13.682730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:13.683715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:13.684720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:13.684720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:13.684720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:13.684720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:13.689401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 289/586] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:13.690657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.693016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a147489-e898-4cf0-b200-628f8fc019a5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.737840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.738522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.738797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.739065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.739905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:13.740208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.741515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:13.741699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.761281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 298/587] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:13.761880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.808470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61f039-be42-4496-a4b7-82e4180e8012 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.845371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.846056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.846339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.846616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.847457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:13.847766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.848830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:13.849305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.868265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 290/588] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:13.869485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.878164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ffca0-339a-4f6a-9af8-e638256aa3bc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:13.915395 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:13.916065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:13.916343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:13.916737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:13.917725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=de86f0a9c2d14cc0b26b19ed719529ef) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:13.918244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:13.919485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:13.925772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 299/589] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/role_assignments?user.id=de86f0a9c2d14cc0b26b19ed719529ef&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:13.926382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:13.991168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.001584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065b17e-5c34-474c-909b-df21c5c367ab None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.046069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.047374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.047921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.048456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.049564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:14.050085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.051742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.052089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51068cce-923b-47d3-a6b6-7f15948dc5e5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.066393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 291/590] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:14.067781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.090865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.091461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.091745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.092056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.106853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.108617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.129506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:14.134615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:14.135678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:14.136749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:14.136749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:14.136749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:14.136749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:14.141462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 300/591] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:14.141462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.189747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7881c1d-2fc3-404b-b380-6d9095a650e0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.234152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.235885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.236201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.236476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.250540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.252770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.270244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61640d-6615-446e-8013-960585acbd0f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:14.272321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:14.274337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:14.276450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:14.276450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:14.276450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:14.276450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:14.276450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 292/592] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:14.276450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.286530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.310224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.311062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.311315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.311591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.320689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.321972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:14.337104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:14.338061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 07:00:14.338998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:14.342907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 301/593] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:14.343901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.399689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002499-41ba-4b1d-a9f3-c97b7cafe1dc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.414149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.442132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.443068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.443499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.443915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.444887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:14.445382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.447013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.460827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 293/594] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:14.462132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.464956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0e505-12b1-4b9b-b44d-9bd2421a8056 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.510058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.510731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.511014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.511284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.512005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:14.512315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.513805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.526095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 302/595] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:14.527348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.558142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.578035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa886d99-f381-41b7-8ab2-87a5d721bb90 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.614476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.615567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.616055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.616524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.617685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:14.618301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.620083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.627612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8ef87-c8bd-4c02-8a0f-9e0167098439 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.640330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 294/596] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:14.641854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.642759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.665386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.666170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.666453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.666722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.674092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.675658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:14.689944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:14.690881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:14.691759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:14.695994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 303/597] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:14.696831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.776556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435df1af-076c-4261-9102-cf0ab9a30375 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.810925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.812335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199045-7de4-49db-8a3d-73a2e06f4f74 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:14.822356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.824244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.824746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.825419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.835825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.838370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.851138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:14.851947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:14.852229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:14.852499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:14.858013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:14.858955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:14.859846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:14.867552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 295/598] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:14.867552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.870197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:14.873100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:14.896265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:14.897250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:14.898159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:14.898159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:14.898159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:14.898159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:14.899016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 304/599] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:14.899220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:14.932242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:14.996100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94c896-08b8-4194-97e4-3198f346717f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.015834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba83f5-4833-4c3a-b13e-193ae9fe95b7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.040361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.041612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.042193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.042646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.044171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:15.044676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.047343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:15.052673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.053451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.053801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.054144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.055036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:15.055406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.056917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:15.062759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 296/600] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:15.063950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.076781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 305/601] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:15.076781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.130214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:15.170137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:15.188722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debe33a-694c-4244-942e-cec351b4fe38 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.195002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a545c-5952-4428-ab87-bf883d2ff7ff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.226914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.228280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.228759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.229266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/b4292042e79c45438b5964710359cd5e/roles/59c2d2f6ca754e289f3766b15292756e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.239426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.240379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.240667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.240943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.241729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:15.242035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.243892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:15.246644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=b4292042e79c45438b5964710359cd5e, role_id=59c2d2f6ca754e289f3766b15292756e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.248428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:15.257339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 297/602] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:15.258384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.266713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 306/603] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:15 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/b4292042e79c45438b5964710359cd5e/roles/59c2d2f6ca754e289f3766b15292756e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:15.267889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.379953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae907b36-6dcb-4b5b-aea2-f4d30300ed19 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.381245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9adef2-358f-46db-a301-b14f3e65cca4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.388063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:15.416783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.417738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.417848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0424d-4703-46cd-ba47-808eb95d8b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.418028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.418288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/366a7e7596d14b32a6ce62790a2e3dd5/roles/59c2d2f6ca754e289f3766b15292756e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.418610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0424d-4703-46cd-ba47-808eb95d8b2a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.418891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0424d-4703-46cd-ba47-808eb95d8b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.419161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0424d-4703-46cd-ba47-808eb95d8b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.422254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:15.427945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0424d-4703-46cd-ba47-808eb95d8b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.430186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0424d-4703-46cd-ba47-808eb95d8b2a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:15.436005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=366a7e7596d14b32a6ce62790a2e3dd5, role_id=59c2d2f6ca754e289f3766b15292756e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.437805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7bc0424d-4703-46cd-ba47-808eb95d8b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:15.451020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:15.452336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:15.453783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:15.455960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 307/604] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:15.457421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e49133-2e15-4d7d-b20d-760e529a43ed None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.457845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e49133-2e15-4d7d-b20d-760e529a43ed None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.458266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e49133-2e15-4d7d-b20d-760e529a43ed None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.459654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 308/605] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:15.463020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 298/606] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:15 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/366a7e7596d14b32a6ce62790a2e3dd5/roles/59c2d2f6ca754e289f3766b15292756e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:15.464828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56958-84c0-492b-8704-e577ebbab60b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.465260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56958-84c0-492b-8704-e577ebbab60b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.465653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56958-84c0-492b-8704-e577ebbab60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.474907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.486339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6ae56958-84c0-492b-8704-e577ebbab60b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:15.488662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6ae56958-84c0-492b-8704-e577ebbab60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:15.588759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 309/607] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 07:00:15.629171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae56958-84c0-492b-8704-e577ebbab60b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.630324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c2784-489c-4bad-93b5-61de920aa383 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.693247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.694331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.694707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.695224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.696418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:15.696823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.698413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:15.715885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 299/608] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:15.721938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.746256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:15.766003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae56958-84c0-492b-8704-e577ebbab60b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.798974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:15.818884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.822137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.822417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.822692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.847215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:15.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:15.869141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:15.870025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 310/609] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:15.874761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:15.891468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f384a64-1098-4466-ac5c-bf18375336de None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:15.956868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:15.958180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:15.958662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:15.959131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/754c137a554941d99f7ef9c380a55ca0/roles/59c2d2f6ca754e289f3766b15292756e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:15.992729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=754c137a554941d99f7ef9c380a55ca0, role_id=59c2d2f6ca754e289f3766b15292756e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:15.994355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:16.030479 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 300/610] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:15 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/754c137a554941d99f7ef9c380a55ca0/roles/59c2d2f6ca754e289f3766b15292756e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:00:16.044843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af720f5d-bd20-447a-a10c-d5d5f6bc33af None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:16.085517 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167c71a-69d3-4ce7-b24e-66f0f70eaca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:16.086198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167c71a-69d3-4ce7-b24e-66f0f70eaca4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.086477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167c71a-69d3-4ce7-b24e-66f0f70eaca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.086750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167c71a-69d3-4ce7-b24e-66f0f70eaca4 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.087577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167c71a-69d3-4ce7-b24e-66f0f70eaca4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:16.087883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167c71a-69d3-4ce7-b24e-66f0f70eaca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:16.089474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167c71a-69d3-4ce7-b24e-66f0f70eaca4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:16.100057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 311/611] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:16.105387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:16.220706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:16.238851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebfc24-7ff8-4929-9aa8-2d5d3dee7e0a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.239204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebfc24-7ff8-4929-9aa8-2d5d3dee7e0a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.239407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebfc24-7ff8-4929-9aa8-2d5d3dee7e0a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.240782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 312/612] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:16.246114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311a5ba-0935-4a28-9633-4acb071de776 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.246465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311a5ba-0935-4a28-9633-4acb071de776 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.246745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311a5ba-0935-4a28-9633-4acb071de776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.267017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac089ac2-e966-422a-8bc9-f3f54e2206dd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:16.268032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c311a5ba-0935-4a28-9633-4acb071de776 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:16.269341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c311a5ba-0935-4a28-9633-4acb071de776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:16.303358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:16.304540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.305022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.305520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.317255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:16.319001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:16.319710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:16.337641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:16.338754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:16.339646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:16.339646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:16.339646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:16.339646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:16.340408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 301/613] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:16.345362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:16.354564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 313/614] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 07:00:16.380469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311a5ba-0935-4a28-9633-4acb071de776 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:16.449121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee19213-68b2-4aec-aea8-71b8ab03794d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:16.481202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:16.507659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:16.507659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.507659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.507659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.508072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:16.508072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:16.508072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:16.528948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 302/615] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:16.534773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:16.635836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311a5ba-0935-4a28-9633-4acb071de776 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:16.702663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0befdb7-3a68-40ab-8545-4e0e7afbf777 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:16.708307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:16.709405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.709690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.710021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.734911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:16.743724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:16.744660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:16.745546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:16.748428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 314/616] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:16.753291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:16.754431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:16.755847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.761716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.762265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.785082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:16.789282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:16.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:16.823265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:16.824657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:16.832004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 303/617] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:00:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:16.842052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:16.843356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:16.911106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde02e7c-2e77-4791-91e0-37a963929eb8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:16.947931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:16.949395 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:16.949936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:16.950403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:16.951338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:16.952010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:16.953785 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:16.955379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:16.966424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 315/618] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:16.973995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:17.086685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:17.091078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1250a-f807-4557-ab0f-cd0af342cbba None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.132892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc90f59-ff02-4e30-8662-a46707513dc1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.145670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.146748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.150756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.151237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.173221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:17.175734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:17.208830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:17.209804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:17.210880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:17.210880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:17.210880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:17.210880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:17.214998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.215904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.216376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.216855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.217914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:17.218441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:17.220069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:17.221759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 316/619] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:17.228774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:17.245418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 304/620] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:00:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:17.251064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:17.348135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:17.355086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ae24-2a5e-4871-acdd-5e49c2e2afff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.394461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.395581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.396102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.396573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.399837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b7cf5-c24a-47ee-862b-86ca4db0c79d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.464077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.464732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.465010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.465301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.466167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:17.466376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:17.467695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:17.473480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:17.475228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:17.497225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 317/621] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:17.510138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:17.533449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:17.578098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:17.585861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 305/622] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 07:00:17.591742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:17.705734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c11889-a215-4268-80f5-22ea9b009685 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.722990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ef5b9-d829-4399-9a14-d09539a68a74 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.751141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.752318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.752813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.753309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.764987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:17.767033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:17.793099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1c4a5-f8e8-4387-a670-0e59fd4321c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.794267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1c4a5-f8e8-4387-a670-0e59fd4321c8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.794763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1c4a5-f8e8-4387-a670-0e59fd4321c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.795238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1c4a5-f8e8-4387-a670-0e59fd4321c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:17.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:17.798761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:17.799705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:17.799705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:17.799705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:17.799705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:17.805788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1c4a5-f8e8-4387-a670-0e59fd4321c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:17.806582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 306/623] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:17.812456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1c4a5-f8e8-4387-a670-0e59fd4321c8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:17.814051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1bb1c4a5-f8e8-4387-a670-0e59fd4321c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:17.830903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:17.831927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:17.832904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:17.834004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 318/624] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:17.839017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e222-9a55-48d2-9864-e3a0a89ed0a5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.839017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e222-9a55-48d2-9864-e3a0a89ed0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.839017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e222-9a55-48d2-9864-e3a0a89ed0a5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.839017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 319/625] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:17.841889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e222-9a55-48d2-9864-e3a0a89ed0a5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:17.880878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:17.919978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:17.934205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00576daf-c2fe-4f7c-adb6-4af20fc2ef5d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.954680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e222-9a55-48d2-9864-e3a0a89ed0a5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:17.970091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca79386-3071-4b69-98f7-1ba672009b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.970837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca79386-3071-4b69-98f7-1ba672009b7e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.971042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca79386-3071-4b69-98f7-1ba672009b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.971319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca79386-3071-4b69-98f7-1ba672009b7e None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.972081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca79386-3071-4b69-98f7-1ba672009b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:17.972386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca79386-3071-4b69-98f7-1ba672009b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:17.973818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca79386-3071-4b69-98f7-1ba672009b7e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:17.990583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 307/626] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:17.990583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eb73c-25cf-44f5-ab35-a8cd1b1c4500 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.990583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eb73c-25cf-44f5-ab35-a8cd1b1c4500 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.990583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eb73c-25cf-44f5-ab35-a8cd1b1c4500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.998528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d70ea6-c954-4aee-8d27-aa3a0100e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:17.998528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d70ea6-c954-4aee-8d27-aa3a0100e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:17.998528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d70ea6-c954-4aee-8d27-aa3a0100e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:17.998528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d70ea6-c954-4aee-8d27-aa3a0100e2f2 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:17.998869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d70ea6-c954-4aee-8d27-aa3a0100e2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:17.998869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d70ea6-c954-4aee-8d27-aa3a0100e2f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.000012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d70ea6-c954-4aee-8d27-aa3a0100e2f2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.011080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c87eb73c-25cf-44f5-ab35-a8cd1b1c4500 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:18.012342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c87eb73c-25cf-44f5-ab35-a8cd1b1c4500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:18.021363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 320/627] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 07:00:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:18.023518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab04c8-77b9-413e-b1a7-8ad02999a486 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.023840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab04c8-77b9-413e-b1a7-8ad02999a486 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.024401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab04c8-77b9-413e-b1a7-8ad02999a486 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.025869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 321/628] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:18.026413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab04c8-77b9-413e-b1a7-8ad02999a486 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.093422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 308/629] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:18.093948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87eb73c-25cf-44f5-ab35-a8cd1b1c4500 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.135527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.151316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab04c8-77b9-413e-b1a7-8ad02999a486 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.181290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.188794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dc58e-9396-4b4b-ba34-2707747c6232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.189779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dc58e-9396-4b4b-ba34-2707747c6232 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.190058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dc58e-9396-4b4b-ba34-2707747c6232 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.190640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dc58e-9396-4b4b-ba34-2707747c6232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.198363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87eb73c-25cf-44f5-ab35-a8cd1b1c4500 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.199619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dc58e-9396-4b4b-ba34-2707747c6232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.202062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dc58e-9396-4b4b-ba34-2707747c6232 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2b4dc58e-9396-4b4b-ba34-2707747c6232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:18.219399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:18.220390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:18.221777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 322/630] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:18.222440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d6acd-f775-4f7e-acc3-608d315bb651 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.222657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d6acd-f775-4f7e-acc3-608d315bb651 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.222902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d6acd-f775-4f7e-acc3-608d315bb651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.238615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.239435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.239708 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.240006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.241144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-570d6acd-f775-4f7e-acc3-608d315bb651 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:18.242331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-570d6acd-f775-4f7e-acc3-608d315bb651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:18.251168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.252628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:18.270967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:18.271920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:18.272799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:18.272799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:18.272799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:18.272799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:18.272799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 309/631] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:18.273755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.325080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 323/632] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:18.325080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d6acd-f775-4f7e-acc3-608d315bb651 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.374077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.385171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f358532-322e-4a0a-a14d-12cf7783932e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.417404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.418421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.418700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.418975 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.424376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d6acd-f775-4f7e-acc3-608d315bb651 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.429507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:18.437497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:18.438646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:18.439756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 310/633] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:18.440790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.450815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.472280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.472887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.473173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.473452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.474229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:18.474557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.476004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.488817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 324/634] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:18.489450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.547455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1d251-71a8-493b-b462-ecd1e527b398 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.580645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdedcd2-fc20-4cc8-894d-521af036edd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.581343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdedcd2-fc20-4cc8-894d-521af036edd2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.581630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdedcd2-fc20-4cc8-894d-521af036edd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.581907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdedcd2-fc20-4cc8-894d-521af036edd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.582663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdedcd2-fc20-4cc8-894d-521af036edd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:18.582971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdedcd2-fc20-4cc8-894d-521af036edd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.584349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdedcd2-fc20-4cc8-894d-521af036edd2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.588626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4414-724f-4f63-9163-9fdb83c2b0ea None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.597473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 311/635] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:18.598586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336a92-5b5b-4fe0-b171-a69bd3643405 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.598870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336a92-5b5b-4fe0-b171-a69bd3643405 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.599141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336a92-5b5b-4fe0-b171-a69bd3643405 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.600353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 312/636] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:18.601033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f336a92-5b5b-4fe0-b171-a69bd3643405 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.613208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.629272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.630183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.630467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.630758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.640882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:18.648521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:18.649588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:18.650448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 325/637] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:18.651389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.694068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.708934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f336a92-5b5b-4fe0-b171-a69bd3643405 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.742811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.743415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.743696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.743976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.744729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:18.745042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.746516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.754456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbae3bf-980e-4ee3-b6ea-6d440b432a9d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.755603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 313/638] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:18.756445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.794563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da33b4-348a-4c35-b13a-ecdf6699852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.795225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da33b4-348a-4c35-b13a-ecdf6699852c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.795502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da33b4-348a-4c35-b13a-ecdf6699852c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.795771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da33b4-348a-4c35-b13a-ecdf6699852c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.796830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da33b4-348a-4c35-b13a-ecdf6699852c None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=95c860dcbf3b40bbb7427a417357c68a) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:18.797286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da33b4-348a-4c35-b13a-ecdf6699852c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.798739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da33b4-348a-4c35-b13a-ecdf6699852c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.804822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 326/639] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/role_assignments?user.id=95c860dcbf3b40bbb7427a417357c68a&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:18.806122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2523918-34d2-4d02-87cf-7634ba8a978f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.806492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2523918-34d2-4d02-87cf-7634ba8a978f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.806772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2523918-34d2-4d02-87cf-7634ba8a978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.825379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b2523918-34d2-4d02-87cf-7634ba8a978f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:18.826640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b2523918-34d2-4d02-87cf-7634ba8a978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:18.847501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.884349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a9e9b-6d06-40cb-8e12-cae9b5d6cfa5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:18.913418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 327/640] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:18.923646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2523918-34d2-4d02-87cf-7634ba8a978f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:18.940548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:18.941952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:18.942242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:18.942655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:18.950741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:18.952221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:18.953319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:18.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:18.970908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 314/641] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:18.971795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.035903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2523918-34d2-4d02-87cf-7634ba8a978f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.071993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.073075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.073436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.074125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.074981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:19.075388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.076870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.089093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 328/642] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:19.089284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1331a-781d-4534-acfd-0f8eacc17cc6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.089385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.119663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.126368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.127394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.127820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.128265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.141656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.143951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:19.166849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:19.168033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:19.169186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:19.169186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:19.169186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:19.169186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:19.170406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 315/643] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:19.171195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.189274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.200594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd26ae-d461-40a7-aa2c-c5c8f862e599 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.236577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.237540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.237831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.238096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.248268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:19.256264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:19.257300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 329/644] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:19.263284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.282875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a627a21-1b8b-4d9d-95ff-fd8b1c1e5b24 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.318691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.319358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.319650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.319905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.320691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:19.320995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.322477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.335629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 316/645] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:19.336450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.372927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61df78-b9a8-4c26-b3de-6fda069cb801 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.386181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.407917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.408716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.408995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.409295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.420302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.421897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:19.439299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:19.440232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:19.441584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:19.441584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:19.441584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:19.441584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:19.441816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc613-5921-4a57-885e-a405f5cb79b8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.441901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 330/646] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:19.441901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.453315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.483188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.483980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.484265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.484539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.485294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:19.485601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.486976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.504295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 317/647] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:19.504295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.553768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdf810-67b8-4b2b-a3e4-ed4bc91258ae None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.590325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.591007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.591286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.591655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.592406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:19.592711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.594161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.602469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 331/648] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:19.603258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.620398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadba144-584d-4c8a-8919-fba0f4e0f900 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.623397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.657935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.658887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.659164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.659471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/de86f0a9c2d14cc0b26b19ed719529ef/roles/59c2d2f6ca754e289f3766b15292756e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.674208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=de86f0a9c2d14cc0b26b19ed719529ef, role_id=59c2d2f6ca754e289f3766b15292756e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.675643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.676409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.697375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 318/649] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:19 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/de86f0a9c2d14cc0b26b19ed719529ef/roles/59c2d2f6ca754e289f3766b15292756e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:19.699307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.714221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233361-f868-44f2-8f7d-8bbeb78b8edd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.757005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.757625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.757906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.758178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.758906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:19.759217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.760601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.783092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 332/650] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:19.783092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.862374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f850c-e947-4288-9a18-19f7eb049dad None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.902968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.904036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.904323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.904598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.914648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.916097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:19.924285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913a94a-789f-4ffa-b5b4-7d7c215b7348 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:19.938937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:19.940121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 07:00:19.941276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:19.942595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 319/651] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:19.947239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:19.963034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.967405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:19.977588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:19.979495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:19.980016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:19.980372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:19.998556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:19.998556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:20.023671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:20.024607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:20.025540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:20.025540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:20.025540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:20.025540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:20.025540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 333/652] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:20.026341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.086297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32105123-0b71-4eff-8e05-c53b5302db86 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.122741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.123595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.123874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.124142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.131663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.133191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:20.148199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:20.149222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:20.150160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:20.154346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 320/653] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:20.155300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.158447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26214f4-9ce5-49df-ab23-5dae4957420a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.196217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.197288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.197805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.198273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.199695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:20.200255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.202083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.221890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 334/654] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:20.223056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.242360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:20.269641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e55e0-4c3c-4792-b3ac-bfec2f690df3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.270913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:20.304705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.305320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.305602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.305875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.306832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:20.307136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.308559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.326477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 321/655] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:20.327633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.337359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8a2e-0112-4f6f-bf00-3c2481f6f361 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.374119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.375352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.375921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.376400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.377516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:20.378029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.379797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.403240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 335/656] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:20.404440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.432471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6075-eed3-4e3b-87fb-b40957b8e0a0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.469634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.470276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.470610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.470883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.473940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:20.512396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:20.517684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31778060-22da-4eca-997e-79d4ffdb02e8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.546233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.547924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.555103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.556260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.556741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.557258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.565726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.567620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:20.584024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:20.585022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 07:00:20.586010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:20.588745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 336/657] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:20.589775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.662166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 322/658] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:20.667729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.695611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43626b2-25af-4917-9450-3f8c98866c42 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.705770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:20.735772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.736814 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.737417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.737889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.768050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:20.782480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672e54-0709-4079-aa5b-19f9efdef9bf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:20.814595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.816610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.821243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:20.821937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:20.822270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:20.822550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:20.823395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:20.823704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:20.825520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:20.844977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 323/659] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:20.845790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.938794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 337/660] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:20.938794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:20.960690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:20.970309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20823962-6e35-4807-b3be-157d5ffa5d13 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.002104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.012755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7333fa-efc2-4178-ac4a-3816be874352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.013599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7333fa-efc2-4178-ac4a-3816be874352 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.013877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7333fa-efc2-4178-ac4a-3816be874352 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.014143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7333fa-efc2-4178-ac4a-3816be874352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.024813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7333fa-efc2-4178-ac4a-3816be874352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.026422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7333fa-efc2-4178-ac4a-3816be874352 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ec7333fa-efc2-4178-ac4a-3816be874352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:21.044373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:21.045310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:21.046257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:21.046257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:21.046257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:21.046257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:21.046516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a5617-9ba0-48d6-ae09-0e9e2ea6818c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.050552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 324/661] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:21.051870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d587f9-b03d-4778-8a31-2eb171644c56 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.052151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d587f9-b03d-4778-8a31-2eb171644c56 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.052423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d587f9-b03d-4778-8a31-2eb171644c56 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.053898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 325/662] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:21.054407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d587f9-b03d-4778-8a31-2eb171644c56 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.086810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a266248-b4fb-4a8e-a177-e37377b1facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.087385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a266248-b4fb-4a8e-a177-e37377b1facf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.087661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a266248-b4fb-4a8e-a177-e37377b1facf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.087937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a266248-b4fb-4a8e-a177-e37377b1facf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.159150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a266248-b4fb-4a8e-a177-e37377b1facf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.160136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d587f9-b03d-4778-8a31-2eb171644c56 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.160800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a266248-b4fb-4a8e-a177-e37377b1facf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.195274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.196074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.196352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.196620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.207791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.209902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.213367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.220931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:21.229734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:21.230629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:21.231539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:21.231539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:21.231539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:21.231539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:21.237399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 326/663] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:21.238378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.271963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 338/664] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:21.273339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606cbb-ca95-4eb8-ac4f-99a2edbcf63b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.273624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606cbb-ca95-4eb8-ac4f-99a2edbcf63b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.273892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606cbb-ca95-4eb8-ac4f-99a2edbcf63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.289701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-84606cbb-ca95-4eb8-ac4f-99a2edbcf63b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:21.290877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-84606cbb-ca95-4eb8-ac4f-99a2edbcf63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:21.347834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dd843-8300-45e9-9ea0-a9642cee3771 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.362695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 339/665] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:21.363422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84606cbb-ca95-4eb8-ac4f-99a2edbcf63b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.391291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.392559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.392843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.393150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.393885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:21.394195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.396029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.408869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 327/666] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:21.409812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.465359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.470678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.479094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84606cbb-ca95-4eb8-ac4f-99a2edbcf63b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.515375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.516155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733a7c-534d-4695-bd67-b1972851412d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.516285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.516476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.516749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.517499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:21.517809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.519254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.531129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 340/667] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:21.531949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.555192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.555904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.556184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.556455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.566889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.568332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:21.589931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:21.590945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:21.591934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:21.591934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:21.591934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:21.591934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:21.591934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 328/668] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:21.592855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.645741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff7c9a-2c10-47bc-bca7-2eb6b9e3302c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.681530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.682341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.682622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.682895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.693428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:21.698621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfab36-ac2b-435e-be7d-03e30ee226f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:21.701445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:21.702460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:21.703428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 341/669] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:21.704191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.716583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.717412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.746055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.746958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.747307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.747648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.754710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.756166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:21.770217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:21.771148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:21.772090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 329/670] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:21.772594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.817797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8f1d0-fde7-47b3-9f03-489b15346b81 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.852341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.853183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.853453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.853721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.860376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.861854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.874876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41d3c-4504-47c5-b842-779d75be306c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:21.876274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:21.877362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 342/671] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:21.878888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.916189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:21.916778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:21.917076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:21.917367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:21.918070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:21.918374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:21.919740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:21.931542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 330/672] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:21.932436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:21.946014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.960899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:21.981067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfcc9c-ae65-465d-970d-7a5d3cd008be None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.015492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaf2b5-9a9c-4bd6-aa93-5bbb548afff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.016172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf2b5-9a9c-4bd6-aa93-5bbb548afff8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.016460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf2b5-9a9c-4bd6-aa93-5bbb548afff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.016737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf2b5-9a9c-4bd6-aa93-5bbb548afff8 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.017532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaf2b5-9a9c-4bd6-aa93-5bbb548afff8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.017842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaf2b5-9a9c-4bd6-aa93-5bbb548afff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.019218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaf2b5-9a9c-4bd6-aa93-5bbb548afff8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.027365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 343/673] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.028572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc694-6dbb-49dd-80d0-7818bca4f3f6 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.028855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc694-6dbb-49dd-80d0-7818bca4f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.029164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc694-6dbb-49dd-80d0-7818bca4f3f6 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.030364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 344/674] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:22.031154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc694-6dbb-49dd-80d0-7818bca4f3f6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.039304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9b428-c5a6-412c-84d7-fee92afc2cb0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.072064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.072721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.073003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.073291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.074439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.074741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.076160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.087887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 331/675] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.088646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.133598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc694-6dbb-49dd-80d0-7818bca4f3f6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.161002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:22.167482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.168088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.168360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.168627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.169439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.169742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.171168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.174069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:22.183199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 345/676] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.183811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.186424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925951-febb-4e01-a997-1ab705bac62e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.225024 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3dcd3-8ccc-477e-b754-5687a479d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.225869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dcd3-8ccc-477e-b754-5687a479d6af None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.226147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dcd3-8ccc-477e-b754-5687a479d6af None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.226418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dcd3-8ccc-477e-b754-5687a479d6af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.233070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3dcd3-8ccc-477e-b754-5687a479d6af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.234688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3dcd3-8ccc-477e-b754-5687a479d6af None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-78b3dcd3-8ccc-477e-b754-5687a479d6af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:22.248936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:22.250432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:22.251462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 332/677] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:22.253911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b4808-c1ba-44fe-a733-4e3fc16b9613 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.254202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b4808-c1ba-44fe-a733-4e3fc16b9613 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.254481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b4808-c1ba-44fe-a733-4e3fc16b9613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.273675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7d0b4808-c1ba-44fe-a733-4e3fc16b9613 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:22.274915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d0b4808-c1ba-44fe-a733-4e3fc16b9613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:22.293678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e96247-91de-42a4-8e8d-25974311b644 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.327029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.327865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.328144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.328410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.338932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.340416 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.340798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 333/678] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:22.341657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b4808-c1ba-44fe-a733-4e3fc16b9613 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:22.370299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:22.371432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:22.372340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:22.372340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:22.372340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:22.372340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:22.372340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 346/679] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:22.372626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.416273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:22.427647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:22.450226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b4808-c1ba-44fe-a733-4e3fc16b9613 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.473718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba4ae4-280c-4b79-b136-fff05092b790 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.485436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.486098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.486381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.486652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.487484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=366a7e7596d14b32a6ce62790a2e3dd5) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.487793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.489264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.495998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 334/680] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/role_assignments?user.id=366a7e7596d14b32a6ce62790a2e3dd5&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.496793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.509403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.510097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.510387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.510684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.511533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=b4292042e79c45438b5964710359cd5e) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.512008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.513625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.521312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 347/681] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/role_assignments?user.id=b4292042e79c45438b5964710359cd5e&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.522194 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.634225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdaa44-44e7-4c86-851a-7f71860dc1e7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.681530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.682131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.682411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.682680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.683497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.683796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.685377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.703814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 335/682] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.704949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.708248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:22.711593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ad7df-876c-4eda-a752-60bb7e325e3e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.722941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:22.762869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.763696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.764052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.764323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.776553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:22.785366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:22.786380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:22.787221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 348/683] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:22.787955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.834460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54fa90-4ff4-4050-ac9d-d15db373a126 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.870662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.871387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.871666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.871936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.872660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.873008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.874555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.887356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 336/684] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.888186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:22.904456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a47e2-e46c-4969-95af-c3e3ae0cded6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:22.954052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:22.954839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:22.955022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:22.961147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:22.961147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=754c137a554941d99f7ef9c380a55ca0) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:22.961147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:22.961147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:22.961555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:22.969425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 349/685] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/role_assignments?user.id=754c137a554941d99f7ef9c380a55ca0&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:22.970496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:23.010308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a9d8-cc52-4865-8b59-cecda8f6ef62 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.051431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.052190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.052469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.052798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.053872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:23.054181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.055994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.067175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 337/686] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:23.072836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:23.074719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:23.171327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30242e1a-1f58-4653-900b-941c113073fd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.213928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.214761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.215038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.215418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.224532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.226009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.237515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d4498-fdd8-4833-9790-fe5ee84825e6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:23.241085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:23.242027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:23.243032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 350/687] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:00:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:23.247991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:23.269453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:23.284246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.285338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.285613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.285887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.300493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.302246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:23.326976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:23.328109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:23.329144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:23.329144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:23.329144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:23.329144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:23.329144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 338/688] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:23.334779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:23.358335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:23.370755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc947d-fe56-4f93-b48b-6e04b5fb8537 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.410645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.411603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.412157 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.412569 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.413629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:23.414076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.415749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.436438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 351/689] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:00:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:23.442521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:23.476954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27484d90-7cef-4e88-b223-27064ee0e79c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.532587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.535158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.535435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.535699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.536423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:23.536726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.538145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.562305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 339/690] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:23.566879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:23.625423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:23.635469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30716d86-3bcc-4329-8408-bc7b20c56303 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.700565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea05b-ec18-421e-97da-5285e3de6c05 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.714698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.715367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.715647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.715921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.753492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:23.764498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.767094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.767094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.767094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.775958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.777902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:23.794737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:23.796054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:23.797958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:23.801100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 340/691] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:23.805520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:23.882761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.884254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.913733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:23.920979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbbdeb-f1de-4fb1-a325-9a9332ecb074 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:23.960247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:23.961368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:23.962007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:23.962523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:23.963860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:23.964392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:23.966063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:23.988274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 341/692] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 07:00:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:23.996669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:24.096428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 352/693] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:24.106049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:24.161349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:24.215170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cca56-ea51-4fd8-941e-2f35a640b1f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:24.220681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf812159-4765-465a-9cf3-67d1c5fc6f97 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:24.273343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:24.274596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:24.275644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:24.276149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:24.291026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:24.292329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:24.292829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:24.293345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:24.298993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:24.302300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:24.343234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:24.344420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:24.345351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:24.345351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:24.345351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:24.345351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:24.345602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:24.345677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 353/694] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:24.349873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:24.425735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:24.427186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:24.537405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:24.542421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122c69-3db7-4cbe-96b3-553ff87844c3 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:24.612519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 342/695] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 07:00:24.617747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:24.625935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:24.626840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:24.627334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:24.627584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:24.628322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:24.628628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:24.630085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:24.646505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 354/696] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:24.653036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:24.687342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:24.760835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce9ce9-d32a-420f-8af0-6ff258b0c205 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:24.799331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2765526-3ab9-4641-8ed2-656bce27b977 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:24.803843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:24.804739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:24.805113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:24.805461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:24.816295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:24.818327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:24.839208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:24.840181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:24.841113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:24.841113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:24.841113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:24.841113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:24.841113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 343/697] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:24.846112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:24.929877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e9936-16b9-41e9-9bd5-a96530a48a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:24.930626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e9936-16b9-41e9-9bd5-a96530a48a14 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:24.930909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e9936-16b9-41e9-9bd5-a96530a48a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:24.931190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e9936-16b9-41e9-9bd5-a96530a48a14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:24.946935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e9936-16b9-41e9-9bd5-a96530a48a14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:24.948409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e9936-16b9-41e9-9bd5-a96530a48a14 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:24.958648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19758947-f1ce-421b-b09a-ba29ca1fc744 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-548e9936-16b9-41e9-9bd5-a96530a48a14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:24.963790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:24.964703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:24.965698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:24.969309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 355/698] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:24.969874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:24.970732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768440-5ef6-449a-969b-064669624bae None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:24.971020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768440-5ef6-449a-969b-064669624bae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:24.971300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768440-5ef6-449a-969b-064669624bae None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:24.973626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 356/699] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:24.979502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d542-30d0-47de-9dd6-f5cb31141dff None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:24.979790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d542-30d0-47de-9dd6-f5cb31141dff None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:24.980098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d542-30d0-47de-9dd6-f5cb31141dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.000586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dfe9d542-30d0-47de-9dd6-f5cb31141dff None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:25.001314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.001837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dfe9d542-30d0-47de-9dd6-f5cb31141dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:25.002458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.002934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.003435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.004520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:25.005089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.007285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.019014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:25.026416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 344/700] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:25.026416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.083000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 357/701] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:25.084482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a724834-40d3-4200-8a4f-04ae818b2b19 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.084823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a724834-40d3-4200-8a4f-04ae818b2b19 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.085199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a724834-40d3-4200-8a4f-04ae818b2b19 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.089101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 358/702] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:25.089101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a724834-40d3-4200-8a4f-04ae818b2b19 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.147755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263bfec-727e-4512-97a5-2c1269a26517 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.184406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba303cd4-f9b1-4b58-b46f-bb57387b9ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.185250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba303cd4-f9b1-4b58-b46f-bb57387b9ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.185451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba303cd4-f9b1-4b58-b46f-bb57387b9ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.185679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba303cd4-f9b1-4b58-b46f-bb57387b9ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.186453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba303cd4-f9b1-4b58-b46f-bb57387b9ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:25.186766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba303cd4-f9b1-4b58-b46f-bb57387b9ee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.188270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba303cd4-f9b1-4b58-b46f-bb57387b9ee9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.198583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a724834-40d3-4200-8a4f-04ae818b2b19 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.200770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 345/703] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:25.202003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a21eb-10fa-4742-814a-aad24121ce2c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.202291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a21eb-10fa-4742-814a-aad24121ce2c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.202595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a21eb-10fa-4742-814a-aad24121ce2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.219813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4f9a21eb-10fa-4742-814a-aad24121ce2c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:25.221078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4f9a21eb-10fa-4742-814a-aad24121ce2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:25.241928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:25.243851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.244664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.245012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.245355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.252610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.254101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:25.268595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:25.269583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:25.270513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 359/704] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:25.271011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.288446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:25.312226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 346/705] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:25.322214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a21eb-10fa-4742-814a-aad24121ce2c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.392102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260b43-e940-438c-b146-1db6c4de852b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.427722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.427947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a21eb-10fa-4742-814a-aad24121ce2c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.428542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.428823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.429126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.429776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.431166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:25.445664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:25.446671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:25.447580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 360/706] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:25.448443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.471310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.471974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.472254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.472525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.473413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=95c860dcbf3b40bbb7427a417357c68a) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:25.473731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.475173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.481715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 347/707] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/role_assignments?user.id=95c860dcbf3b40bbb7427a417357c68a&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:25.494263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.524924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:25.539977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:25.570263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59b491-917b-4557-80a8-e8abce09e49a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.612509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.613251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.613611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.613886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.614814 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:25.615123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.616625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.617574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8b76d-da02-4ffc-8649-7b0fb94e0360 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.633230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 361/708] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:25.635076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.660355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82da672-cda9-4110-b1a7-246ccd33869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.661201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82da672-cda9-4110-b1a7-246ccd33869b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.661481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82da672-cda9-4110-b1a7-246ccd33869b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.661753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82da672-cda9-4110-b1a7-246ccd33869b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.673837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82da672-cda9-4110-b1a7-246ccd33869b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.675294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82da672-cda9-4110-b1a7-246ccd33869b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d82da672-cda9-4110-b1a7-246ccd33869b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:25.692859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:25.694030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:25.695001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:25.695001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:25.695001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:25.695001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:25.695001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 348/709] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:25.695333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91868270-c44b-4da2-a651-82f2e3b307d3 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.695583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91868270-c44b-4da2-a651-82f2e3b307d3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.695914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91868270-c44b-4da2-a651-82f2e3b307d3 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.701752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 349/710] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:25.702682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91868270-c44b-4da2-a651-82f2e3b307d3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.749637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4bf9b-83ff-434c-96fc-ca54ee369d75 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.779451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:25.787294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.787964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.788244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.788512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.789254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:25.789553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.790952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.792149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:25.800680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 362/711] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:25.800680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:25.819209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91868270-c44b-4da2-a651-82f2e3b307d3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.859752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1890e1-d4fb-4b32-9e91-b34c9489f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.860489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1890e1-d4fb-4b32-9e91-b34c9489f48f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.860729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1890e1-d4fb-4b32-9e91-b34c9489f48f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.861000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1890e1-d4fb-4b32-9e91-b34c9489f48f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.861846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1890e1-d4fb-4b32-9e91-b34c9489f48f None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=de86f0a9c2d14cc0b26b19ed719529ef) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:25.862159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1890e1-d4fb-4b32-9e91-b34c9489f48f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.863584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1890e1-d4fb-4b32-9e91-b34c9489f48f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.876084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 350/712] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/role_assignments?user.id=de86f0a9c2d14cc0b26b19ed719529ef&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:25.876084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35640949-0063-4c25-a718-9cdbff796d14 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.876084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35640949-0063-4c25-a718-9cdbff796d14 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.876084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35640949-0063-4c25-a718-9cdbff796d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.895752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-35640949-0063-4c25-a718-9cdbff796d14 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:25.897640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-35640949-0063-4c25-a718-9cdbff796d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:25.915034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f620247-7f2a-492d-80b0-09d761a5323e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:25.960739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:25.962549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:25.963054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:25.963541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:25.965003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:25.965658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:25.968049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:25.991012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 363/713] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:25.991012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:26.004302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 351/714] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:26.006152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35640949-0063-4c25-a718-9cdbff796d14 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:26.083845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:26.107838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:26.136566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76976310-edc2-44fd-a645-cee3f405ddcd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:26.152375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35640949-0063-4c25-a718-9cdbff796d14 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:26.174451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:26.175134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:26.175752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:26.175752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:26.176399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:26.176759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:26.178144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:26.184424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 364/715] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 07:00:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:26.185394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:26.190673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:26.191368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:26.191652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:26.191928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:26.192646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:26.194171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:26.218137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 352/716] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:00:26 2022] POST /identity/v3/users => generated 254 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:26.334251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9bf98-f57f-46e7-a95c-f35a99d212f0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:26.367424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:26.379753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:26.380578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:26.380855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:26.381154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:26.381826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:26.384080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:26.396521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:26.397693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:26.398565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 365/717] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:26 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:26.403952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:26.520697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4557d-ff67-425d-b879-db77b50d5cf0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:26.545406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:26.560492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547a414-94e8-47e8-8e18-236a78e8f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:26.561307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a414-94e8-47e8-8e18-236a78e8f9bf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:26.561656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a414-94e8-47e8-8e18-236a78e8f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:26.562051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a414-94e8-47e8-8e18-236a78e8f9bf None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:26.562842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547a414-94e8-47e8-8e18-236a78e8f9bf None admin] RBAC: Adding query filter params (name=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:26.563210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547a414-94e8-47e8-8e18-236a78e8f9bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:26.564672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547a414-94e8-47e8-8e18-236a78e8f9bf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:26.570999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 353/718] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:26.576852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:26.701810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a553261-79dc-4a4f-803e-3c919d3779b1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:26.751810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043f3ac-2448-408f-bad1-a6114e7128a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:26.752592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043f3ac-2448-408f-bad1-a6114e7128a7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:26.752943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043f3ac-2448-408f-bad1-a6114e7128a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:26.753300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043f3ac-2448-408f-bad1-a6114e7128a7 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:26.754086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043f3ac-2448-408f-bad1-a6114e7128a7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:26.754513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043f3ac-2448-408f-bad1-a6114e7128a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:26.756027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043f3ac-2448-408f-bad1-a6114e7128a7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:26.764977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 366/719] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:27.758631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39943-2fb9-4964-bf9f-91ecf4b7d7c1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:27.759185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39943-2fb9-4964-bf9f-91ecf4b7d7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:27.759185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39943-2fb9-4964-bf9f-91ecf4b7d7c1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:27.760857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 354/720] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:27.767673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a5e6-0533-4c37-997d-31c82458cd9a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:27.768238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a5e6-0533-4c37-997d-31c82458cd9a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:27.768238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a5e6-0533-4c37-997d-31c82458cd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:27.794145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-09e0a5e6-0533-4c37-997d-31c82458cd9a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:27.795358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-09e0a5e6-0533-4c37-997d-31c82458cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:27.855023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:27.887712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 367/721] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:27.922269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae39943-2fb9-4964-bf9f-91ecf4b7d7c1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:28.083772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae39943-2fb9-4964-bf9f-91ecf4b7d7c1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:28.134286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:28.135101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:28.135382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:28.135654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:28.150556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:28.160755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:28.161764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:28.162606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 355/722] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:28.168470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a5e6-0533-4c37-997d-31c82458cd9a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:28.295638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a5e6-0533-4c37-997d-31c82458cd9a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:28.340625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8574ff-5497-41c5-841f-5368d0115b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:28.341251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8574ff-5497-41c5-841f-5368d0115b73 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:28.341539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8574ff-5497-41c5-841f-5368d0115b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:28.341811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8574ff-5497-41c5-841f-5368d0115b73 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:28.342580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8574ff-5497-41c5-841f-5368d0115b73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:28.342890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8574ff-5497-41c5-841f-5368d0115b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:28.344882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8574ff-5497-41c5-841f-5368d0115b73 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:28.354348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 368/723] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:28.359852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:28.450155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:28.525111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6e1ef-727d-471c-8e46-a188d546cb99 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:28.536616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fc2b8-edc4-4d17-adf5-c0f47badb399 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:28.536925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fc2b8-edc4-4d17-adf5-c0f47badb399 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:28.537233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fc2b8-edc4-4d17-adf5-c0f47badb399 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:28.538878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 369/724] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:28.543702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92eb9-5226-47d1-b0ac-f97aa33e16b1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:28.543990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92eb9-5226-47d1-b0ac-f97aa33e16b1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:28.544268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92eb9-5226-47d1-b0ac-f97aa33e16b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:28.571803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-79b92eb9-5226-47d1-b0ac-f97aa33e16b1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:28.574722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-79b92eb9-5226-47d1-b0ac-f97aa33e16b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:28.585747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:28.586619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:28.586905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:28.587211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:28.603589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:28.605033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:28.632723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:28.633734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:28.634645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:28.634645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:28.634645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:28.634645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:28.639420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 356/725] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:28.646141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:28.649707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:28.713729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 370/726] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:28.750350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b92eb9-5226-47d1-b0ac-f97aa33e16b1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:28.779519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:28.784589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f4779-b2e2-4f0b-a41e-646d6d1ef9d4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:28.820048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:28.821108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:28.821621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:28.822096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:28.823207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:28.824060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:28.825683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:28.837974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 357/727] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:28.845080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:28.937477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b92eb9-5226-47d1-b0ac-f97aa33e16b1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:28.979837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec212ca-2ef5-4c10-8429-de96c0afad8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:28.980966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec212ca-2ef5-4c10-8429-de96c0afad8d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:28.981436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec212ca-2ef5-4c10-8429-de96c0afad8d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:28.981719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec212ca-2ef5-4c10-8429-de96c0afad8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:28.993091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d8a0-de40-4a16-9d31-2e82cd6d03a8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:28.994389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec212ca-2ef5-4c10-8429-de96c0afad8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7ec212ca-2ef5-4c10-8429-de96c0afad8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:29.002414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:29.003382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 371/728] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:29.009380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383f67-e9a7-4899-af85-6ff789b743c3 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.010136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383f67-e9a7-4899-af85-6ff789b743c3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.010136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383f67-e9a7-4899-af85-6ff789b743c3 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.011076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 372/729] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:29.012345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53936-3525-4638-bf32-cf8c9312bf10 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.012632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53936-3525-4638-bf32-cf8c9312bf10 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.012907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef53936-3525-4638-bf32-cf8c9312bf10 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.017351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 373/730] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:29.017351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef53936-3525-4638-bf32-cf8c9312bf10 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.039100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e1f10-553b-4bbc-bc04-e8753fb625ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.040061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1f10-553b-4bbc-bc04-e8753fb625ea None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.040340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1f10-553b-4bbc-bc04-e8753fb625ea None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.040608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1f10-553b-4bbc-bc04-e8753fb625ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.048445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:29.048565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e1f10-553b-4bbc-bc04-e8753fb625ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:29.050029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e1f10-553b-4bbc-bc04-e8753fb625ea None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-828e1f10-553b-4bbc-bc04-e8753fb625ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:29.065476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:29.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:29.067456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:29.069393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 358/731] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:00:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:29.070969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02744293-ebae-461a-8bf1-2decbfb1286f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.071464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02744293-ebae-461a-8bf1-2decbfb1286f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.071738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02744293-ebae-461a-8bf1-2decbfb1286f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.089570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-02744293-ebae-461a-8bf1-2decbfb1286f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:29.090761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-02744293-ebae-461a-8bf1-2decbfb1286f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:29.104447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:29.128482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef53936-3525-4638-bf32-cf8c9312bf10 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:29.173074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 359/732] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:29.173074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae636-ad96-44ca-b51d-c1c6de2f7f8a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.173074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae636-ad96-44ca-b51d-c1c6de2f7f8a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.173074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae636-ad96-44ca-b51d-c1c6de2f7f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.182405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a8eae636-ad96-44ca-b51d-c1c6de2f7f8a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:29.184568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a8eae636-ad96-44ca-b51d-c1c6de2f7f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:29.187666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.189306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.189589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.189864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.190622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:29.190929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:29.192329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:29.204177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 374/733] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:29.204983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.257923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 360/734] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:29.259080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eae636-ad96-44ca-b51d-c1c6de2f7f8a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.322307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:29.330327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ce064-94f0-4e5d-8f15-a6e3b363d5c6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:29.348915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:29.362790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.363464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.363743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.364021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.364855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:29.365197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:29.366655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:29.370092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eae636-ad96-44ca-b51d-c1c6de2f7f8a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:29.383904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 375/735] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:29.384694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.403883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.404568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.404848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.405152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.405981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:29.407337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:29.422374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 361/736] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:00:29 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:29.423177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.509514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664101ba-7a7d-4156-afcb-d8e9eea249f1 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:29.552663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ac45-a9c8-4e05-b1fe-631bd6129e53 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:29.554212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cabfc5-cef8-4dc5-8cc9-9364cae7c117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.555140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cabfc5-cef8-4dc5-8cc9-9364cae7c117 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.555424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cabfc5-cef8-4dc5-8cc9-9364cae7c117 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.555701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cabfc5-cef8-4dc5-8cc9-9364cae7c117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.567371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cabfc5-cef8-4dc5-8cc9-9364cae7c117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:29.568896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cabfc5-cef8-4dc5-8cc9-9364cae7c117 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:29.580541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-22cabfc5-cef8-4dc5-8cc9-9364cae7c117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:29.590634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:29.591603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:29.592499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:29.592499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:29.592499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:29.592499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:29.592499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 376/737] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:29.592827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc72f-fe23-49c3-af9e-87932aaee47c None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.593238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc72f-fe23-49c3-af9e-87932aaee47c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.593882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc72f-fe23-49c3-af9e-87932aaee47c None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.595412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 377/738] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:29.596256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc72f-fe23-49c3-af9e-87932aaee47c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.599475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.601248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.601539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.601818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.604028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:29.616271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:29.624272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:29.625422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:29.626301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:29.628397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 362/739] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:29.629391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.714447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc72f-fe23-49c3-af9e-87932aaee47c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:29.736794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a678cc-bf36-47d8-bac1-b902cd3ceb3e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:29.754576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d131e-f9b1-40dd-8d34-b39473b3a374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.755774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d131e-f9b1-40dd-8d34-b39473b3a374 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.756212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d131e-f9b1-40dd-8d34-b39473b3a374 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.756592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d131e-f9b1-40dd-8d34-b39473b3a374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.770315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d131e-f9b1-40dd-8d34-b39473b3a374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:29.771577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d131e-f9b1-40dd-8d34-b39473b3a374 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:29.776227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:29.776902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.777215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.777473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.778208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:29.778530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:29.780295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-748d131e-f9b1-40dd-8d34-b39473b3a374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:29.793570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:29.794574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:29.795519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:29.795519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:29.795519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:29.795519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:29.799612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 378/740] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:29.801522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010855-5002-4cba-a5a2-1d0a057f3307 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:29.801809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010855-5002-4cba-a5a2-1d0a057f3307 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:29.802089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86010855-5002-4cba-a5a2-1d0a057f3307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:29.821293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-86010855-5002-4cba-a5a2-1d0a057f3307 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:29.822532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-86010855-5002-4cba-a5a2-1d0a057f3307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:29.879209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:29.912720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 379/741] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 07:00:29.912720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86010855-5002-4cba-a5a2-1d0a057f3307 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.914794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 363/742] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:29.915573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:29.976911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.027955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86010855-5002-4cba-a5a2-1d0a057f3307 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.029318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b589749-d834-4d6a-a3c3-2246683d906b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.062466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.063163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.063477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.063721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.064464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:30.064817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.066212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.069386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.070363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.070780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.071183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.072852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:30.073348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.080261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 380/743] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:30.080430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.080512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.092230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 364/744] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:30.097382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.131816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.190727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d0110-4354-4a85-b4cd-235608968722 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.200749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c205a0d-65aa-463f-ac45-c13fe6640de9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.215467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.228748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.229613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.229895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.230166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.236912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.237422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.237608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.237876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.238146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.238853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.238947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.240197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.253928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 365/745] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:00:30 2022] POST /identity/v3/services => generated 240 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:30.254250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:30.255205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:30.256170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 381/746] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:30.256788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.261034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.386084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01b401-a455-4aa8-a53a-0e24a1834eab None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.408108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec976b-edfe-4ef2-bf80-89c07514dcd6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.411541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.437590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.438544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.438890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.439230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.451616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.451978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.452473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.452763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.453033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.453651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.460051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.461591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:30.471580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:30.472492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:30.473816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:30.473816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:30.473816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:30.473816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:30.473816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 366/747] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:30.474103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:30.478394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:30.479275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:30.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 382/748] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:30.480660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.513435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.594394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec449-82dc-4242-964f-d306b29db186 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.598486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b254271-9e91-4d95-964e-6fbd0d28a84c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.632989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.633638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.633918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.634192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.634958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:30.635224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.636656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.643798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.645259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.645777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.646664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.647971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:30.648491 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.650136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.653833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 383/749] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:30.654070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.676879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 367/750] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:30.676879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.686624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.771388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a86f6-f680-418e-83f8-0673351b5d05 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.776629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.799549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b06a33-a99e-420f-98f9-92aa1b68c0cf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.819666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.820660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.821163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.822223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.823468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:30.824343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.826877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.836888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:30.837750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:30.838029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:30.838300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:30.848476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 384/751] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:30.849533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:30.849871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.851480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:30.868329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:30.869534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:30.870679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:30.877536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 368/752] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:30.878500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:30.962353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:30.983504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2caa8b-908e-4f4d-ac44-6161ff74fb6e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:30.993871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a938c09-7ba3-4958-bbcb-1dee58719751 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:31.025451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:31.026835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.027357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.027786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.034499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:31.035525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.035902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.036270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/95c860dcbf3b40bbb7427a417357c68a/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.038361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=95c860dcbf3b40bbb7427a417357c68a, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:31.040005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:31.052054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:31.062170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 369/753] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:30 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/95c860dcbf3b40bbb7427a417357c68a/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:31.062170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:31.193700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf0424-0214-4700-a629-4079a6db9949 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:31.195529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:31.198536 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:31.235899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:31.236615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.236960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.237350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.238826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:31.239250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:31.240918 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:31.264113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 370/754] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 07:00:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:31.270006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:31.342064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:31.381706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:31.406407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad515940-9cc7-4dae-a3d9-8c1a262391bd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:31.412573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 385/755] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:00:31.414564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:31.457529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913aa815-c72f-4837-aea6-9cf13c464a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:31.458094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aa815-c72f-4837-aea6-9cf13c464a09 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.458380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aa815-c72f-4837-aea6-9cf13c464a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.458655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aa815-c72f-4837-aea6-9cf13c464a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.546469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb576-a11d-4ac0-8194-2e5028826ba2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:31.548173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913aa815-c72f-4837-aea6-9cf13c464a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:31.549872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913aa815-c72f-4837-aea6-9cf13c464a09 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:31.592239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199c8c5-0bcf-45bb-b60f-1f95caef6d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:31.593194 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c8c5-0bcf-45bb-b60f-1f95caef6d60 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.593537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c8c5-0bcf-45bb-b60f-1f95caef6d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.593971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c8c5-0bcf-45bb-b60f-1f95caef6d60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.608396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199c8c5-0bcf-45bb-b60f-1f95caef6d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:31.611102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199c8c5-0bcf-45bb-b60f-1f95caef6d60 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:31.612334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-4199c8c5-0bcf-45bb-b60f-1f95caef6d60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:31.634956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:31.635960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:31.636851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:31.636851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:31.636851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:31.636851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:31.636851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 386/756] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:31.642104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c79ff-1eab-459e-a2eb-73907fbaee0e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.642385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c79ff-1eab-459e-a2eb-73907fbaee0e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.642658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c79ff-1eab-459e-a2eb-73907fbaee0e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.644098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 387/757] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:31.644647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c79ff-1eab-459e-a2eb-73907fbaee0e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:31.680955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 371/758] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 07:00:31.682373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2a308-f4e2-4d4b-a4a1-97b01f709c7b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.682657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2a308-f4e2-4d4b-a4a1-97b01f709c7b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.682924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2a308-f4e2-4d4b-a4a1-97b01f709c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.693896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:31.703748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5cc2a308-f4e2-4d4b-a4a1-97b01f709c7b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:31.705078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5cc2a308-f4e2-4d4b-a4a1-97b01f709c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:31.777080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 372/759] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:31.778183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2a308-f4e2-4d4b-a4a1-97b01f709c7b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:31.783830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c79ff-1eab-459e-a2eb-73907fbaee0e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:31.827838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:31.828888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.829388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.829859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.830798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:31.831820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:31.833879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:31.850155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 388/760] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:31.855739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:31.889094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:31.903510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2a308-f4e2-4d4b-a4a1-97b01f709c7b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:31.940455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:31.941303 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:31.941581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:31.941848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:31.953778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:31.955474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:31.970077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:31.976071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee7691-cad1-4f6c-a912-65e775a96bbe None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:31.976236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:31.977480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:31.978594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:31.978594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:31.978594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:31.978594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:31.981330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 373/761] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:31.982258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.028490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.029752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.030254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.030762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.031708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.033450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:32.047451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:32.048459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:32.049386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:32.052496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 389/762] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Jul 12 07:00:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:32.053645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.102545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caeec3-2df8-40b3-b6ce-6ed8526864d6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.138335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.139186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.139464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.139735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.147560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.148934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:32.164210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72101026-03b3-4ab8-86ab-5799447a0257 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:32.166796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:32.167681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:32.168585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:32.172749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 374/763] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:32.173643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.205343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.206808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.207288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.207781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.208698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:32.209315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.211080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.229083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 390/764] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:32.230415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.254448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:32.281232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b253f-c61b-4c73-95d4-3bc847001a98 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.320847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.321819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.322281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.322709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.323584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:32.324398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.326012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.336947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 375/765] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:32.338271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.407394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:32.454869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b8cd2-2f06-429e-9988-98ef7f47a3f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.473007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497982b6-4a73-464b-8211-a7208791b1ab None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.491772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.492693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.492965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.493251 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.500209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.501810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.509639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1947-a8e8-4e71-a6c6-c73d4f17cdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.510302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1947-a8e8-4e71-a6c6-c73d4f17cdaa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.510579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1947-a8e8-4e71-a6c6-c73d4f17cdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.510844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1947-a8e8-4e71-a6c6-c73d4f17cdaa None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.511624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1947-a8e8-4e71-a6c6-c73d4f17cdaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:32.511923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1947-a8e8-4e71-a6c6-c73d4f17cdaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.513366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1947-a8e8-4e71-a6c6-c73d4f17cdaa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:32.516276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:32.517187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:32.518080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 376/766] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:32.519076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.530293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 391/767] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:32.531534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de506-316e-4757-9ee7-01213af2d397 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.531813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de506-316e-4757-9ee7-01213af2d397 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.532082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de506-316e-4757-9ee7-01213af2d397 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.533443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 392/768] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:32.533984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de506-316e-4757-9ee7-01213af2d397 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.619128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:32.641141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400021-65b4-4132-ad9c-da2e0e00116a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.641724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de506-316e-4757-9ee7-01213af2d397 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.664721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:32.675809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.676478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.676957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.677086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.677734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:32.678037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.679402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.681567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01f504-a79d-447b-a710-a02bbcf72c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.682231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01f504-a79d-447b-a710-a02bbcf72c81 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.682507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01f504-a79d-447b-a710-a02bbcf72c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.682775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01f504-a79d-447b-a710-a02bbcf72c81 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.683545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01f504-a79d-447b-a710-a02bbcf72c81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:32.683847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01f504-a79d-447b-a710-a02bbcf72c81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.690473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 377/769] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:32.690604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01f504-a79d-447b-a710-a02bbcf72c81 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.690682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.701496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 393/770] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:32.702609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42262e-ea03-437d-aec9-6de018f8c81e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.702892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42262e-ea03-437d-aec9-6de018f8c81e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.703165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42262e-ea03-437d-aec9-6de018f8c81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.719835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-df42262e-ea03-437d-aec9-6de018f8c81e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:32.721021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-df42262e-ea03-437d-aec9-6de018f8c81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:32.791116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 394/771] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:32.792368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42262e-ea03-437d-aec9-6de018f8c81e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.815788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00ed5b-d4ea-4769-9456-5ee02cd32aff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.859398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.860453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.860988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.861296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.862143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=de86f0a9c2d14cc0b26b19ed719529ef) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:32.862456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.864710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:32.867232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:32.877081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 378/772] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/role_assignments?user.id=de86f0a9c2d14cc0b26b19ed719529ef&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:32.877081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:32.923265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42262e-ea03-437d-aec9-6de018f8c81e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:32.945706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:32.979105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:32.979891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:32.980484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:32.980825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:32.982230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] RBAC: Adding query filter params (interface=public, service_id=94144115377348e4a63da0277c4f7ab3, region_id=RegionOne) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:32.982812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:32.984633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:33.005367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 395/773] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/endpoints?service_id=94144115377348e4a63da0277c4f7ab3&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:33.006709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:33.017113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2cee1-ee69-496c-89b8-b7a227cd53de None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:33.062668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:33.064031 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.064788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.065286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.066565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=d901a025bdee47e4800f8a250bef1245) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:33.067052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:33.069661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:33.078081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 379/774] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:32 2022] GET /identity/v3/role_assignments?user.id=d901a025bdee47e4800f8a250bef1245&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:33.079475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:33.190315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8f822-0a8d-475a-a798-5060abb66f3c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:33.227192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:33.228177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.228627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.229725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] PATH_INFO: `/v3/services/network` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.229725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:33.231242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:33.233301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:33.244950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:33.246277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:33.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:33.249542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 396/775] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:33.255299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:33.292289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b35ebf-4156-419c-af14-f8c47ab3f76d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:33.327326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:33.363987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a46f9e-4ff2-4440-8ef9-53622d42a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:33.364664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a46f9e-4ff2-4440-8ef9-53622d42a328 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.364949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a46f9e-4ff2-4440-8ef9-53622d42a328 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.365230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a46f9e-4ff2-4440-8ef9-53622d42a328 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.365912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a46f9e-4ff2-4440-8ef9-53622d42a328 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:33.367278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a46f9e-4ff2-4440-8ef9-53622d42a328 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:33.376026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 380/776] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/services => generated 772 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:33.391109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313bdd7-c8b8-498a-a18a-231ec1a55eec None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.391322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313bdd7-c8b8-498a-a18a-231ec1a55eec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.391602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313bdd7-c8b8-498a-a18a-231ec1a55eec None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.397178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 381/777] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:33.397871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1f03f-760f-447e-a50a-7e4f0c27bd73 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:33.399164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf406bd-f139-4779-87b6-13fe0d3eec8c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.399454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf406bd-f139-4779-87b6-13fe0d3eec8c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.399903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf406bd-f139-4779-87b6-13fe0d3eec8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.422881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3cf406bd-f139-4779-87b6-13fe0d3eec8c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:33.424155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3cf406bd-f139-4779-87b6-13fe0d3eec8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:33.443418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:33.444170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.444452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.444720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.445468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] RBAC: Adding query filter params (name=network) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:33.446256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:33.447984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:33.455730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 397/778] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:33.461470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:33.505196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 382/779] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:00:33.525879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf406bd-f139-4779-87b6-13fe0d3eec8c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:33.533297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:33.580922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eaa61-4ca7-4b3b-ad4a-d599cb1b2dda None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:33.622495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:33.623168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.623447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.623866 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.624926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] RBAC: Adding query filter params (type=network) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:33.625280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:33.626671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:33.634932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 398/780] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:33.640815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:33.683314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf406bd-f139-4779-87b6-13fe0d3eec8c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:33.706452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:33.758597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ec812-82f0-488c-b4a3-12741f111b78 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:33.774998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:33.776196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.777757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.778187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.798103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:33.798325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:33.810907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:33.811576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:33.811797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:33.812923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:33.814193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:33.814881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:33.814881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:33.814881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] RBAC: Adding query filter params (interface=public, service_id=577d96b5427343108107b46ef94989eb, region_id=RegionOne) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:33.815059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 383/781] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:33.815127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:33.815127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:33.825174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 399/782] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/endpoints?service_id=577d96b5427343108107b46ef94989eb&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:33.826026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:33.826136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:34.017257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7defc-2635-4f3e-addd-7368b415fa5a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:34.020893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f75d8b-e83f-4668-9bd8-53529108e5fc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:34.080086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:34.080770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:34.081072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:34.081335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:34.082026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:34.083404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:34.095702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:34.096378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:34.096656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:34.096927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:34.104043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:34.104459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:34.105915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:34.106555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 384/783] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:34.116148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 400/784] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:34.122109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:34.166756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:34.262650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8efaa8-39e3-4e6e-b743-8180b05652cc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:34.310037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:34.310848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:34.311128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:34.311402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:34.328068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:34.329661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:34.347920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:34.348834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:34.350233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:34.350233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:34.350233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:34.350233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:34.350233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 385/785] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:34.355012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:34.452089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:34.515085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719362ab-7a1e-4e3a-8449-f59ff546b69f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:34.560138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:34.560812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:34.561165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:34.561440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:34.562621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:34.562924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:34.564307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:34.582064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 401/786] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:34.587216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:34.674029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:34.696807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c0b6-a29e-49c5-9d97-8f9dd95d6c35 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:34.743964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:34.744783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:34.745073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:34.745349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:34.753231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:34.754672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:34.776004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:34.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:34.778595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 386/787] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:00:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:34.789282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:34.909507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63837e-d792-4fdf-8a15-fab340c272d4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:34.923190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:34.963531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38764811-647a-4671-9837-eb79b00cbac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:34.964226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764811-647a-4671-9837-eb79b00cbac9 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:34.964500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764811-647a-4671-9837-eb79b00cbac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:34.964770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764811-647a-4671-9837-eb79b00cbac9 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:34.965644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38764811-647a-4671-9837-eb79b00cbac9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:34.965945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38764811-647a-4671-9837-eb79b00cbac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:34.967375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38764811-647a-4671-9837-eb79b00cbac9 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:34.996103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 402/788] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:00:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:35.004219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:35.109906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a346f-4bd6-47dc-b5ab-1f248a1c125d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:35.161785 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:35.162457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.162734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.163003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.179238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:35.220658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483597d-963e-4a8c-a7c0-e715853f52c0 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.221338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483597d-963e-4a8c-a7c0-e715853f52c0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.221841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483597d-963e-4a8c-a7c0-e715853f52c0 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.224748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 403/789] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:35.229376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d863b1-1860-41d5-be03-4b2c10c9e57e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.229924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d863b1-1860-41d5-be03-4b2c10c9e57e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.230397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d863b1-1860-41d5-be03-4b2c10c9e57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.244276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:35.245956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:35.269789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-13d863b1-1860-41d5-be03-4b2c10c9e57e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:35.271379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-13d863b1-1860-41d5-be03-4b2c10c9e57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:35.378535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 404/790] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:35.401770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 387/791] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:00:35.407672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d863b1-1860-41d5-be03-4b2c10c9e57e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:35.419288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:35.492480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:35.593655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:35.601346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d863b1-1860-41d5-be03-4b2c10c9e57e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:35.605849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb3607-92b3-4ff6-982b-d91167f6a0cc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:35.636318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5b9b0-d571-4166-8546-7c4cc30cc315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:35.637070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5b9b0-d571-4166-8546-7c4cc30cc315 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.637349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5b9b0-d571-4166-8546-7c4cc30cc315 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.637613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5b9b0-d571-4166-8546-7c4cc30cc315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.649113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a90ef1-f379-4ff4-9ae6-4a7fd276258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:35.649676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5b9b0-d571-4166-8546-7c4cc30cc315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:35.650577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a90ef1-f379-4ff4-9ae6-4a7fd276258b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.651059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a90ef1-f379-4ff4-9ae6-4a7fd276258b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.651164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5b9b0-d571-4166-8546-7c4cc30cc315 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:35.651624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a90ef1-f379-4ff4-9ae6-4a7fd276258b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.668107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a90ef1-f379-4ff4-9ae6-4a7fd276258b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-43f5b9b0-d571-4166-8546-7c4cc30cc315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:35.670199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:35.671084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:35.672909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:35.672909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:35.672909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:35.672909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:35.672909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 405/792] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:35.677457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e084-a985-4f9b-9307-1c1252d8b99d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.677786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e084-a985-4f9b-9307-1c1252d8b99d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.678070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e084-a985-4f9b-9307-1c1252d8b99d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.683483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 406/793] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:35.683483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e084-a985-4f9b-9307-1c1252d8b99d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-14a90ef1-f379-4ff4-9ae6-4a7fd276258b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:35.686329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:35.688118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:35.688974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:35.696709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 388/794] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:35.696709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c1f13-cbd2-43a9-9c00-936f5ab7caaa None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.696709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c1f13-cbd2-43a9-9c00-936f5ab7caaa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.696709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c1f13-cbd2-43a9-9c00-936f5ab7caaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.724652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-952c1f13-cbd2-43a9-9c00-936f5ab7caaa None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:35.726270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-952c1f13-cbd2-43a9-9c00-936f5ab7caaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:35.802305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e084-a985-4f9b-9307-1c1252d8b99d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:35.825987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 389/795] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:35.826852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c1f13-cbd2-43a9-9c00-936f5ab7caaa None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:35.858029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:35.861548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:35.861830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:35.862100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:35.863129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:35.863435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:35.865473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:35.877457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:35.885345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 407/796] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:35.886913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:35.906420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:35.970769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c1f13-cbd2-43a9-9c00-936f5ab7caaa None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:35.998447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f97578-fd29-4c83-974f-26c6bd576939 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.016744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.017965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.018387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.019275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.020436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:36.020884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.023062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.032636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 390/797] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:36.033687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.042138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605116e-a27f-4baf-8340-056d69da66e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.042968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605116e-a27f-4baf-8340-056d69da66e6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.043252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605116e-a27f-4baf-8340-056d69da66e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.043526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605116e-a27f-4baf-8340-056d69da66e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.054434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605116e-a27f-4baf-8340-056d69da66e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c605116e-a27f-4baf-8340-056d69da66e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:36.062372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:36.063301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:36.064151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:36.068577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 408/798] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:36.069964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07bdf4-3a49-4c4d-984e-7898bc5369f2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.070255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07bdf4-3a49-4c4d-984e-7898bc5369f2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.070538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07bdf4-3a49-4c4d-984e-7898bc5369f2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.071853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 409/799] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:36.072538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07bdf4-3a49-4c4d-984e-7898bc5369f2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.142529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.146856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685b57-6e9b-4ea8-88c3-c0986da91a82 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.149131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.185629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.186501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.186809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.187256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.187836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07bdf4-3a49-4c4d-984e-7898bc5369f2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.194380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.195834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:36.211002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:36.211953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:36.212941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 391/800] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:36.213472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.229679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163a04-0ec8-4c3a-95ec-eb6b51243e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.230480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163a04-0ec8-4c3a-95ec-eb6b51243e9b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.230781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163a04-0ec8-4c3a-95ec-eb6b51243e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.231041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163a04-0ec8-4c3a-95ec-eb6b51243e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.241438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163a04-0ec8-4c3a-95ec-eb6b51243e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.242900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163a04-0ec8-4c3a-95ec-eb6b51243e9b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d8163a04-0ec8-4c3a-95ec-eb6b51243e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:36.260803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:36.261935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:36.262838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:36.262838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:36.262838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:36.262838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:36.262838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 410/801] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:36.263849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236de74a-4183-4f3d-baf5-4011b3e66878 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.263849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236de74a-4183-4f3d-baf5-4011b3e66878 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.263849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236de74a-4183-4f3d-baf5-4011b3e66878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.281627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-236de74a-4183-4f3d-baf5-4011b3e66878 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:36.282815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-236de74a-4183-4f3d-baf5-4011b3e66878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:36.324827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee2e6e-e073-4039-a790-c875e2248b7e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.352463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 411/802] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:36.353164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236de74a-4183-4f3d-baf5-4011b3e66878 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.375484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.376164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.376446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.376718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.377499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:36.377818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.379165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.391908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 392/803] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:36.391908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.411446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.416600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.464350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236de74a-4183-4f3d-baf5-4011b3e66878 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.493120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87679b83-fc97-4ebc-a47a-e9995470b85c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.500183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.500868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.501178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.501459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.502250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:36.502561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.504002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.516387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 412/804] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:36.517240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.534948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.535617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.535902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.536176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.536901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:36.537252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.538839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.551297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 393/805] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:36.551297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.625963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce4aeb-a4b0-4c7b-9f56-729788d2fdd6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.635187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.640309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.654822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a004f32-842e-44e0-9e91-4936f7d943b4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.661125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.662007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.662227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.662502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.663150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.664528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:36.676773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:36.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:36.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 413/806] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:36.679851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.693783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.694592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.694868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.695138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.706043 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.707470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:36.724355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:36.725346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:36.726258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:36.726258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:36.726258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:36.726258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:36.726258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 394/807] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:36.726542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.785965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263fad-9ab3-441a-b0b9-a7b0dac49727 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.820811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec604a1-a11b-468d-b3eb-069c33771456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.821506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec604a1-a11b-468d-b3eb-069c33771456 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.821788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec604a1-a11b-468d-b3eb-069c33771456 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.822061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec604a1-a11b-468d-b3eb-069c33771456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.822896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec604a1-a11b-468d-b3eb-069c33771456 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=95c860dcbf3b40bbb7427a417357c68a) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:36.823214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec604a1-a11b-468d-b3eb-069c33771456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.823386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e1d83-5d47-404c-98b2-d9219da109df None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:36.825013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec604a1-a11b-468d-b3eb-069c33771456 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.830919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 414/808] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/role_assignments?user.id=95c860dcbf3b40bbb7427a417357c68a&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:36.832167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6f902-88e7-412d-9ed7-1de2323dec96 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.832421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6f902-88e7-412d-9ed7-1de2323dec96 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.832711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6f902-88e7-412d-9ed7-1de2323dec96 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.833955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 415/809] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:36.834671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6f902-88e7-412d-9ed7-1de2323dec96 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.860447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.867249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:36.868502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:36.868752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:36.869074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:36.878412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:36.879894 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:36.896673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:36.898039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:36.899877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:36.901185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:36.901294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 395/810] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:36.902196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:36.977123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6f902-88e7-412d-9ed7-1de2323dec96 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.023032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7051c-a54e-4f4a-940f-719c8e1b0e14 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.024356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c668858-ffab-46c0-bc09-4e980683a42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.025376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c668858-ffab-46c0-bc09-4e980683a42e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.025376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c668858-ffab-46c0-bc09-4e980683a42e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.025668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c668858-ffab-46c0-bc09-4e980683a42e None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.026370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c668858-ffab-46c0-bc09-4e980683a42e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:37.026677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c668858-ffab-46c0-bc09-4e980683a42e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.028060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c668858-ffab-46c0-bc09-4e980683a42e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.032876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 416/811] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:37.034836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999bb90-a89d-40fa-937d-b48d425e8b4d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.034836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999bb90-a89d-40fa-937d-b48d425e8b4d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.034836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999bb90-a89d-40fa-937d-b48d425e8b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.053494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1999bb90-a89d-40fa-937d-b48d425e8b4d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:37.054695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1999bb90-a89d-40fa-937d-b48d425e8b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:37.065147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.065832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.066117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.066507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.067242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:37.067550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.068940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.082760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 396/812] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:37.083599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.137209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 417/813] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:37.138150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999bb90-a89d-40fa-937d-b48d425e8b4d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.149416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.179534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.190748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f0087-2831-4221-8b59-95c5497fd877 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.231508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.232557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.233089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.233582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.235325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:37.235890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.237492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.251960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999bb90-a89d-40fa-937d-b48d425e8b4d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.257688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 397/814] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:37.258510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.289298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.289993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.290264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.290541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.291253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:37.291558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.292939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.303438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 418/815] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:37.303438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.367513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec836b23-53eb-4c76-ac62-4ef21ff2dd35 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.379167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.403986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.404825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.405117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.405397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.407955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.411374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ccfc8-963e-48bb-9de9-fa1a41bcfa6e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.413732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.415186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:37.430157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:37.431308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:37.432522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:37.436131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 398/816] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:37.437144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.451798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.453720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.454278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.454718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] PATH_INFO: `/v3/services/network` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.455596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.457410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:37.470311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:37.471283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 07:00:37.472235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:37.474314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 419/817] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:37.475183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.543480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217c6d4-bb8b-46a5-831f-b1bb6c375eff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.577620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.578427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.578715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.578988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.581796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f468fe1-b91c-41f4-a609-f32143e3417c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.589706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.591165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:37.608962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:37.610082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:37.611172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:37.611172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:37.611172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:37.611172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:37.611172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 399/818] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:37.611172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.624404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d745b-7255-4be5-882b-311646af003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.625369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d745b-7255-4be5-882b-311646af003d None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.625834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d745b-7255-4be5-882b-311646af003d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.626331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d745b-7255-4be5-882b-311646af003d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.627215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d745b-7255-4be5-882b-311646af003d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.628757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d745b-7255-4be5-882b-311646af003d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.635445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.658745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 420/819] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:00:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:37.658745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d745b-7255-4be5-882b-311646af003d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.687018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.744096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7ad06-991a-48c8-af7d-40b7e98a2da6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.792020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d745b-7255-4be5-882b-311646af003d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.796854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.797854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.798328 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.798750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.799746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:37.800285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.803136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.823322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 400/820] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:37.824855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.847111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:37.848085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:37.848376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:37.848654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:37.849411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] RBAC: Adding query filter params (name=network) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:37.849720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:37.851100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:37.857557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 421/821] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:37.858367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:37.920998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.950766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcb727-3e4f-4ce4-9d73-b4eb1ee646da None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:37.969865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:37.993401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799850a-cbe8-4211-b902-b9cf5b165a3c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.000728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.001535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.001830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.002113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.002859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:38.003172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.004572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.017978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 401/822] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:38.018927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.034377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.035692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.036217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.036678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.048812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.050727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:38.070475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:38.071421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:38.072315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:38.072315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:38.072315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:38.072315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:38.074991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 422/823] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:38.076103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.127903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6de0a-e30d-4fca-a9ec-1ffa8de4161e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.162784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.163554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.163739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.164017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.164728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:38.165037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.166461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.168593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:38.172644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 402/824] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:38.173542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.193431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173754-bd05-4dce-8f10-156a9a03acc7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.225765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:38.231923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.232904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.233360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.233863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.241276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.242945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:38.257719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:38.258817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:38.259728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:38.262023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 423/825] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:38.263006 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.287934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffa1a-bc61-4e04-b21d-d22ec68a5fd8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.332023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.332616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.332933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.333687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.334429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:38.334874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.336560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.352542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 403/826] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:38.353579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.372466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b6ad0-eed0-455d-87ee-c4c1b6088245 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.416421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:38.417956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.419052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.419665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.420149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.421304 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.422635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.451544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 424/827] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:00:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:38.452918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.467728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115133-254a-45ab-95d8-86d3ac8e40f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.525509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:38.525823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.527437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.527901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.528347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.529391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:38.529881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.532638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.550400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 404/828] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:38.557189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.593564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fe658-4993-486a-9cbc-e8513a3d1a8a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.631623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.632692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.633148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.633561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.641409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.643106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:38.658651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:38.659771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:38.660960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:38.663193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 425/829] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:38.668754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:38.774467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731576a-55bb-4472-94fe-f95ec755aadb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.794832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:38.811465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45023fb4-3398-44a9-9df4-bd5d7df3d635 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:38.816477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.817598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.818021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.818460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/de86f0a9c2d14cc0b26b19ed719529ef/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.821819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:38.824538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=de86f0a9c2d14cc0b26b19ed719529ef, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.826175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.843944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 405/830] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:38 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/de86f0a9c2d14cc0b26b19ed719529ef/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:38.851099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf6acd-33e2-45f5-8849-50c85b217186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:38.851850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf6acd-33e2-45f5-8849-50c85b217186 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:38.852128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf6acd-33e2-45f5-8849-50c85b217186 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:38.852401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf6acd-33e2-45f5-8849-50c85b217186 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:38.853309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf6acd-33e2-45f5-8849-50c85b217186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:38.853621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf6acd-33e2-45f5-8849-50c85b217186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:38.855165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf6acd-33e2-45f5-8849-50c85b217186 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:38.869921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 426/831] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:38.875640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:39.009576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24742c1-2e74-4daf-a51a-5415e74a870a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:39.027990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ad782-0815-4c24-965b-3033337b81a1 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.028209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ad782-0815-4c24-965b-3033337b81a1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.028476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ad782-0815-4c24-965b-3033337b81a1 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.029889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 427/832] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:39.035624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ddbe7-4d13-4f47-8d20-cd1f95c5e388 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.035911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ddbe7-4d13-4f47-8d20-cd1f95c5e388 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.036189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ddbe7-4d13-4f47-8d20-cd1f95c5e388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.048305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:39.049471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.049971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.050385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/d901a025bdee47e4800f8a250bef1245/roles/59c2d2f6ca754e289f3766b15292756e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.061078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5f4ddbe7-4d13-4f47-8d20-cd1f95c5e388 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:39.061078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5f4ddbe7-4d13-4f47-8d20-cd1f95c5e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:39.066317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=d901a025bdee47e4800f8a250bef1245, role_id=59c2d2f6ca754e289f3766b15292756e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:39.067772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:39.088179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 406/833] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:38 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/d901a025bdee47e4800f8a250bef1245/roles/59c2d2f6ca754e289f3766b15292756e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:39.144903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 428/834] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:39.168018 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:39.199530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:39.291911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58d478-2d93-431c-b310-f3ef91293526 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:39.325811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:39.326647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.326934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.327206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.327900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:39.329308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:39.343727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:39.344735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:39.345631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 407/835] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Jul 12 07:00:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:39.350957 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ddbe7-4d13-4f47-8d20-cd1f95c5e388 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:39.452252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:39.457614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ddbe7-4d13-4f47-8d20-cd1f95c5e388 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:39.495752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2fbe9-7579-404b-a549-757967797288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:39.496427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2fbe9-7579-404b-a549-757967797288 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.496707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2fbe9-7579-404b-a549-757967797288 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.496976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2fbe9-7579-404b-a549-757967797288 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.497716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2fbe9-7579-404b-a549-757967797288 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:39.498020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2fbe9-7579-404b-a549-757967797288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:39.499461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2fbe9-7579-404b-a549-757967797288 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:39.504639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 429/836] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:39.509990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:39.621748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480e69-d581-4df0-98d4-9b5596040146 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:39.649446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:39.658018 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139187df-035b-4e5a-8498-a706bcfca7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:39.658792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139187df-035b-4e5a-8498-a706bcfca7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.659080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139187df-035b-4e5a-8498-a706bcfca7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.665089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139187df-035b-4e5a-8498-a706bcfca7b8 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.665089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139187df-035b-4e5a-8498-a706bcfca7b8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:39.665089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139187df-035b-4e5a-8498-a706bcfca7b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:39.665089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139187df-035b-4e5a-8498-a706bcfca7b8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:39.667151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 408/837] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:39.892273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbb3e8-8da3-4453-ac84-46dc5daf09aa None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.892451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbb3e8-8da3-4453-ac84-46dc5daf09aa None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.892744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbb3e8-8da3-4453-ac84-46dc5daf09aa None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.894363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 430/838] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:39.899787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452837c0-cc39-457a-8e59-ce9d19b6b05a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:39.900072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452837c0-cc39-457a-8e59-ce9d19b6b05a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:39.900345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452837c0-cc39-457a-8e59-ce9d19b6b05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:39.917520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-452837c0-cc39-457a-8e59-ce9d19b6b05a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:39.918705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-452837c0-cc39-457a-8e59-ce9d19b6b05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:39.987470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 409/839] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:40.012100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbb3e8-8da3-4453-ac84-46dc5daf09aa None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:40.119046 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbb3e8-8da3-4453-ac84-46dc5daf09aa None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:40.153977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:40.154707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:40.154990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:40.155265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:40.156190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:40.157615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:40.169849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:40.172153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:40.174145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 431/840] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:40.181127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452837c0-cc39-457a-8e59-ce9d19b6b05a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:40.262289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:40.280069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452837c0-cc39-457a-8e59-ce9d19b6b05a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:40.321715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b5fc-406c-4e66-8fd9-5f7cf73207d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:40.322397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b5fc-406c-4e66-8fd9-5f7cf73207d5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:40.322683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b5fc-406c-4e66-8fd9-5f7cf73207d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:40.322957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b5fc-406c-4e66-8fd9-5f7cf73207d5 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:40.324022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b5fc-406c-4e66-8fd9-5f7cf73207d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:40.324329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b5fc-406c-4e66-8fd9-5f7cf73207d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:40.325750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b5fc-406c-4e66-8fd9-5f7cf73207d5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:40.331238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 410/841] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:40.338206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:40.367449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:40.444942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b2b0-d770-4bc2-b258-e740164c0383 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:40.484094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d375098-91bd-45f4-bf8e-081fe6c8a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:40.484831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d375098-91bd-45f4-bf8e-081fe6c8a637 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:40.485043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d375098-91bd-45f4-bf8e-081fe6c8a637 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:40.485328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d375098-91bd-45f4-bf8e-081fe6c8a637 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:40.486043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d375098-91bd-45f4-bf8e-081fe6c8a637 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:40.486348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d375098-91bd-45f4-bf8e-081fe6c8a637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:40.487719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d375098-91bd-45f4-bf8e-081fe6c8a637 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:40.492699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 432/842] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:41.015539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354f7c-653d-415e-b458-c1ba66e6e77e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.015941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354f7c-653d-415e-b458-c1ba66e6e77e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.016123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354f7c-653d-415e-b458-c1ba66e6e77e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.017801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 411/843] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:41.022989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367b563-2a14-424e-ac03-ec22dbe698a4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.023277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367b563-2a14-424e-ac03-ec22dbe698a4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.023559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367b563-2a14-424e-ac03-ec22dbe698a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.040473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8367b563-2a14-424e-ac03-ec22dbe698a4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:41.041682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8367b563-2a14-424e-ac03-ec22dbe698a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:41.109632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 433/844] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:41.132791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1354f7c-653d-415e-b458-c1ba66e6e77e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.223337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69f2c2-eda6-4bd0-b01c-3b8f295d0a06 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.223792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69f2c2-eda6-4bd0-b01c-3b8f295d0a06 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.224204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69f2c2-eda6-4bd0-b01c-3b8f295d0a06 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.226546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 434/845] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:41.234102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e5828-606e-40a0-a3b4-8d5dbb0ac71b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.234388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e5828-606e-40a0-a3b4-8d5dbb0ac71b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.234665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e5828-606e-40a0-a3b4-8d5dbb0ac71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.246624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1354f7c-653d-415e-b458-c1ba66e6e77e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.254021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7b9e5828-606e-40a0-a3b4-8d5dbb0ac71b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:41.255848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7b9e5828-606e-40a0-a3b4-8d5dbb0ac71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:41.282556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:41.283255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:41.283410 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.283641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.283916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.295088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:41.303681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:41.304874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:00:41.305955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:41.309220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 412/846] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:41.310980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.326842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:41.349084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 435/847] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:41.370241 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e5828-606e-40a0-a3b4-8d5dbb0ac71b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.430754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95c1b0-0a9f-441f-a462-3a331deecbf4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.470541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:41.471753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.472231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.472798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.473734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:41.474048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:41.475541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:41.485089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 413/848] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:41.491237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.499875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e5828-606e-40a0-a3b4-8d5dbb0ac71b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.543077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:41.544165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.544453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.544733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.557825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:41.568305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:41.569566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:00:41.570410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:41.573113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 436/849] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:41.575791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.589413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:41.592420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:41.615544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7987a-6957-4597-afc7-d02b99c7f55e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.658315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:41.659189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.659463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.659822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.671376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:41.673900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:41.692888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:41.693884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:41.694787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:41.694787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:41.694787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:41.694787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:41.699374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 414/850] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:41.700364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.716544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca2b9c-5bbf-488e-bd55-6309f5e6a08f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.762153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:41.762744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.764138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.764419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.765397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:41.765993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:41.767464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:41.780806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 437/851] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:41.782198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.822624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00077e2e-1ac6-47f1-acfc-8b32d51bc5b3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.846576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:41.858449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdfe0e-3fec-4998-af96-c971231d42a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:41.859164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdfe0e-3fec-4998-af96-c971231d42a9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.859519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdfe0e-3fec-4998-af96-c971231d42a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.859791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdfe0e-3fec-4998-af96-c971231d42a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.860517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdfe0e-3fec-4998-af96-c971231d42a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:41.860820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdfe0e-3fec-4998-af96-c971231d42a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:41.862238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdfe0e-3fec-4998-af96-c971231d42a9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:41.863203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:41.874153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 415/852] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:41.875644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371ea12-1f5d-4abe-8ea5-fdc33ca63258 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.875928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371ea12-1f5d-4abe-8ea5-fdc33ca63258 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.876199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371ea12-1f5d-4abe-8ea5-fdc33ca63258 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.881091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 416/853] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:41.881091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371ea12-1f5d-4abe-8ea5-fdc33ca63258 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:41.904030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1b540-a068-4c02-976c-dd1f5d248155 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.941650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4901b5a-e13a-4975-be37-4790f0ab38c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:41.942449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901b5a-e13a-4975-be37-4790f0ab38c5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.942722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901b5a-e13a-4975-be37-4790f0ab38c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.943003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901b5a-e13a-4975-be37-4790f0ab38c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.954401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4901b5a-e13a-4975-be37-4790f0ab38c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:41.955850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4901b5a-e13a-4975-be37-4790f0ab38c5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d4901b5a-e13a-4975-be37-4790f0ab38c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:41.973254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:41.974167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:41.975067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:41.975067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:41.975067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:41.975067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:41.975067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 438/854] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:41.975886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976fec1-0f7e-4996-9c0e-a39398a73b64 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:41.975886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976fec1-0f7e-4996-9c0e-a39398a73b64 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:41.976426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976fec1-0f7e-4996-9c0e-a39398a73b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:41.989334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371ea12-1f5d-4abe-8ea5-fdc33ca63258 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:41.998087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c976fec1-0f7e-4996-9c0e-a39398a73b64 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:41.999766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c976fec1-0f7e-4996-9c0e-a39398a73b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:42.026581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.027314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.027596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.027868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.035377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.036819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:42.052674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:42.053706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:42.054662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 417/855] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:00:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:42.055273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.078553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 439/856] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:42.079716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976fec1-0f7e-4996-9c0e-a39398a73b64 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.114934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:42.135872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:42.179520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eca815-e07b-40f7-87bc-d73767053fb0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:42.182416 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976fec1-0f7e-4996-9c0e-a39398a73b64 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:42.215310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.216097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.216379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.216659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.217410 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:42.217715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.219184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.222196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.222879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.223193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.223505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.224349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:42.224659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.226108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.236832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 418/857] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:42.236832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.254062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 440/858] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 07:00:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:42.254062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.343163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fb43-0eb9-4a72-b861-8aa70832132c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:42.344644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:42.363955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3c167-b74e-46a4-a2a0-243b57dc0fc0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:42.378596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:42.392490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.393278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.393554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.393823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.394524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.395979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.403023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.403980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.404624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.404914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.412818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.414394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.415015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 419/859] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:00:42 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:42.415549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:42.445025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:42.446171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:42.447168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:42.451938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 441/860] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:00:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:42.453448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.560152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff677f7e-99eb-4ee2-8e84-ce1df4d8e07c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:42.566100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb6f4f-55ef-4d21-8b46-c9ded57817e7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:42.601908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9346a0-35bd-417f-a41c-3c57633fe33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.602813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9346a0-35bd-417f-a41c-3c57633fe33b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.602813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9346a0-35bd-417f-a41c-3c57633fe33b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.603102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9346a0-35bd-417f-a41c-3c57633fe33b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.603346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e9f9e-5ae9-4eb8-883d-0946544e5bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.604026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e9f9e-5ae9-4eb8-883d-0946544e5bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.604309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e9f9e-5ae9-4eb8-883d-0946544e5bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.604583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e9f9e-5ae9-4eb8-883d-0946544e5bb1 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.605797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e9f9e-5ae9-4eb8-883d-0946544e5bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:42.605797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e9f9e-5ae9-4eb8-883d-0946544e5bb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.607891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e9f9e-5ae9-4eb8-883d-0946544e5bb1 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.626182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 442/861] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 07:00:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:42.627563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888722-9875-49c3-9d46-2f24c3fb16e9 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.627853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888722-9875-49c3-9d46-2f24c3fb16e9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.628131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888722-9875-49c3-9d46-2f24c3fb16e9 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.633892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 443/862] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:42.634891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:42.635023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7888722-9875-49c3-9d46-2f24c3fb16e9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.672581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:42.681176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9346a0-35bd-417f-a41c-3c57633fe33b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.682644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9346a0-35bd-417f-a41c-3c57633fe33b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.761603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7888722-9875-49c3-9d46-2f24c3fb16e9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:42.800769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 420/863] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:42 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:00:42.801791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:42.802457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.802741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.803017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.803368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3cbb-6c5c-460f-aabc-3a5d309c45ca None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:42.803854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3cbb-6c5c-460f-aabc-3a5d309c45ca None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:42.804318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3cbb-6c5c-460f-aabc-3a5d309c45ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:42.823257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-450e3cbb-6c5c-460f-aabc-3a5d309c45ca None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:42.824874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-450e3cbb-6c5c-460f-aabc-3a5d309c45ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:42.877740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:42.879509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:42.881886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:42.900454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 421/864] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:42 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:42.901330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e3cbb-6c5c-460f-aabc-3a5d309c45ca None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:42.925096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:43.001610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 444/865] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:42 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:00:43.002452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.029437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e3cbb-6c5c-460f-aabc-3a5d309c45ca None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.066587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.067654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.068073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.068602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.081933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.083710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.103469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491fbec-aeaf-4139-bf5b-f38c63e7367d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:43.103941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:43.104933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:43.106061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:43.106061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:43.106061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:43.106061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:43.112501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 422/866] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:43.113615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.136109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:43.142692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.143579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.143689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.143911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.144635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.146059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.173077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 445/867] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:00:43 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:43.173077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.176220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:43.238229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c67a8-0b70-4d68-8ce9-94f7f5cd86f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.290221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.291167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.291449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.291731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.308260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.310764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.318050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b132d39-0cdd-443d-a4d1-d513c9a91e28 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:43.337966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:43.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:43.339818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:43.339818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:43.339818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:43.339818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:43.344636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 423/868] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:43.346324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.367335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.368339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.368812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.369328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.370273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:43.370670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.372138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.388756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 446/869] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:43.395988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.453170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:43.471976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:43.489749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e15a22-7779-494c-a704-be00ddf9f280 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.508034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8914eb7-57f3-412e-9551-7e066e2ecefa None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.534104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.534912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.535260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.535597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.536398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:43.536764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.538243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.546047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.547271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.547790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.548261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.555266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 424/870] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:43.555896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.556587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.557804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:43.572461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:43.574257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:43.575196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:43.578144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 447/871] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:43.586793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.672767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf69bfd-db39-4a3c-bc31-a254fcb1f1ef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.708335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b447709-2df8-4cbb-9496-63c7c99eb504 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.720674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:43.721777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.722874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.723154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.723434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.733647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.735797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.746857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.747601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.748011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.748351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.749367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:43.749751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.751248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:43.753509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:43.754558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:43.755722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:43.760030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 425/872] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:43.761521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.767645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 448/873] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:43.773303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.775076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:43.869671 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7954c3d-16f8-44bb-98c7-fecc38579d65 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.882694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e901ed6-fc2d-4380-9106-45b0f8a91b25 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:43.907257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.908352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.908933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.909478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.910472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:43.910980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.912626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.919343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da63d76-c2ff-4a7f-bc89-4b7353f5e5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:43.920014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da63d76-c2ff-4a7f-bc89-4b7353f5e5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:43.920296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da63d76-c2ff-4a7f-bc89-4b7353f5e5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:43.920575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da63d76-c2ff-4a7f-bc89-4b7353f5e5c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:43.922329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da63d76-c2ff-4a7f-bc89-4b7353f5e5c7 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=de86f0a9c2d14cc0b26b19ed719529ef) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:43.922329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da63d76-c2ff-4a7f-bc89-4b7353f5e5c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:43.923466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da63d76-c2ff-4a7f-bc89-4b7353f5e5c7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:43.928425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 426/874] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:43.934875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:43.941245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 449/875] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/role_assignments?user.id=de86f0a9c2d14cc0b26b19ed719529ef&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:44.017424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:44.055797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272b8c7-a02d-44cc-a940-8fbebc5f6cb1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:44.093253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39c2ef-2013-4dc7-9091-aeb7e56c6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:44.094065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39c2ef-2013-4dc7-9091-aeb7e56c6b1f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:44.094319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39c2ef-2013-4dc7-9091-aeb7e56c6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:44.094542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39c2ef-2013-4dc7-9091-aeb7e56c6b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:44.095387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39c2ef-2013-4dc7-9091-aeb7e56c6b1f None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=d901a025bdee47e4800f8a250bef1245) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:44.095700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39c2ef-2013-4dc7-9091-aeb7e56c6b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:44.097181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39c2ef-2013-4dc7-9091-aeb7e56c6b1f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:44.104494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 427/876] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:43 2022] GET /identity/v3/role_assignments?user.id=d901a025bdee47e4800f8a250bef1245&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:44.538359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fdfa8-8047-4828-9b66-f3b901579262 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:44.538781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fdfa8-8047-4828-9b66-f3b901579262 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:44.538960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fdfa8-8047-4828-9b66-f3b901579262 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:44.540574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 450/877] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:44.545819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba137b11-e7f2-4fa4-99c2-de2e7cca2097 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:44.547110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba137b11-e7f2-4fa4-99c2-de2e7cca2097 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:44.547110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba137b11-e7f2-4fa4-99c2-de2e7cca2097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:44.567557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ba137b11-e7f2-4fa4-99c2-de2e7cca2097 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:44.569120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba137b11-e7f2-4fa4-99c2-de2e7cca2097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:44.641300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 428/878] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:44.663746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fdfa8-8047-4828-9b66-f3b901579262 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:44.701934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:44.774194 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fdfa8-8047-4828-9b66-f3b901579262 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:44.809774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:44.810514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:44.810790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:44.811060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:44.811741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:44.813223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:44.826262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:44.827215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:44.828826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 451/879] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Jul 12 07:00:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:44.833787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba137b11-e7f2-4fa4-99c2-de2e7cca2097 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:44.897627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:44.939213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba137b11-e7f2-4fa4-99c2-de2e7cca2097 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:44.975361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:44.976038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:44.976323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:44.976600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:44.977368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:44.977368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:44.979836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:44.985826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 429/880] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:45.002053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:45.102371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:45.106856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cf919-4c77-491e-ac9b-590bb6c8924f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:45.142210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284d352-be32-4f7e-9e8e-91bd35b0203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:45.142892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d352-be32-4f7e-9e8e-91bd35b0203e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.143174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d352-be32-4f7e-9e8e-91bd35b0203e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.143453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d352-be32-4f7e-9e8e-91bd35b0203e None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.144165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284d352-be32-4f7e-9e8e-91bd35b0203e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:45.144471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284d352-be32-4f7e-9e8e-91bd35b0203e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:45.145879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284d352-be32-4f7e-9e8e-91bd35b0203e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:45.151332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 452/881] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:45.158189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:45.268303 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00351047-48cf-411f-936a-41cfe0d4a70e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:45.298369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:45.307337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:45.308316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.308960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.309686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.310490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] RBAC: Adding query filter params (interface=public, service_id=ae37acd943e8407a8f22b50368b196f7, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:45.310803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:45.313456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:45.321863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 430/882] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:00:45 2022] GET /identity/v3/endpoints?service_id=ae37acd943e8407a8f22b50368b196f7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:45.322024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27efe34-d63b-4017-bb22-4031c312a731 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.322254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27efe34-d63b-4017-bb22-4031c312a731 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.322523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27efe34-d63b-4017-bb22-4031c312a731 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.324347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 453/883] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:45.330075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:45.330323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc00c84-6fdf-4e57-8321-fd18c5ffe708 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.332575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc00c84-6fdf-4e57-8321-fd18c5ffe708 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.332575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc00c84-6fdf-4e57-8321-fd18c5ffe708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.352492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7cc00c84-6fdf-4e57-8321-fd18c5ffe708 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:45.353663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7cc00c84-6fdf-4e57-8321-fd18c5ffe708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:45.436423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 454/884] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:45.451253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6a782-e5b0-4d98-b6e4-c3cefa6db93f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:45.459674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc00c84-6fdf-4e57-8321-fd18c5ffe708 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:45.489678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:45.491176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.491724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.492215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.493125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:45.494707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:45.503029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 431/885] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:00:45 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:00:45.559121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:45.568744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc00c84-6fdf-4e57-8321-fd18c5ffe708 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:45.608243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:45.609084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.609376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.609670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.610369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:45.611775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:45.624524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:45.625929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:45.626824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 455/886] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:45.632301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:45.695808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:45.745479 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e8000-354c-451e-a99e-9e2967f109d6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:45.781162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:45.781845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.782128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.782405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.783119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:45.783427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:45.784807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:45.790214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 432/887] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:45.795488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:45.899021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfb445-d890-4f4a-828f-5468aea65c81 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:45.933353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:45.934028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:45.934306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:45.934576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:45.935279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:45.935582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:45.937084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:45.942417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 456/888] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:45.948161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.063755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d503ca1-8f86-4a5f-8471-a2a0af61e6a5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:46.082358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:46.099797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff5332-8f21-4297-b5ec-cb96a3c3da6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:46.100810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5332-8f21-4297-b5ec-cb96a3c3da6f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.101345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5332-8f21-4297-b5ec-cb96a3c3da6f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.101847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5332-8f21-4297-b5ec-cb96a3c3da6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.102832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5332-8f21-4297-b5ec-cb96a3c3da6f None admin] RBAC: Adding query filter params (interface=public, service_id=4edeb67d23b64ba2bc530bc9d5ef4445, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:46.103339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5332-8f21-4297-b5ec-cb96a3c3da6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:46.104904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5332-8f21-4297-b5ec-cb96a3c3da6f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:46.110799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 433/889] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:00:45 2022] GET /identity/v3/endpoints?service_id=4edeb67d23b64ba2bc530bc9d5ef4445&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:46.119555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.145471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:46.199546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365bbc6-4020-4270-b961-8ab86d8a4344 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.199832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365bbc6-4020-4270-b961-8ab86d8a4344 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.200113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365bbc6-4020-4270-b961-8ab86d8a4344 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.201834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 434/890] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:46.207113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2e1e2-d274-4071-a981-231045ac7d56 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.207400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2e1e2-d274-4071-a981-231045ac7d56 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.207681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2e1e2-d274-4071-a981-231045ac7d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.221212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea5498-1937-43cf-b437-afecf7dd2402 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:46.225608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cab2e1e2-d274-4071-a981-231045ac7d56 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:46.226806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cab2e1e2-d274-4071-a981-231045ac7d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:46.259793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9174db-e450-485a-96a1-9ff3d9ce073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:46.260471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9174db-e450-485a-96a1-9ff3d9ce073b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.260754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9174db-e450-485a-96a1-9ff3d9ce073b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.261030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9174db-e450-485a-96a1-9ff3d9ce073b None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.261743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9174db-e450-485a-96a1-9ff3d9ce073b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:46.263133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9174db-e450-485a-96a1-9ff3d9ce073b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:46.273097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 457/891] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:00:46 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:00:46.273097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fecd2d-c6d0-4228-a0be-c0820df07e8f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.273097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fecd2d-c6d0-4228-a0be-c0820df07e8f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.273097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fecd2d-c6d0-4228-a0be-c0820df07e8f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.273097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 458/892] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:46.283069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab04e35-dfc4-4010-83c1-980ad4c3b6dc None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.283352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab04e35-dfc4-4010-83c1-980ad4c3b6dc None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.283619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab04e35-dfc4-4010-83c1-980ad4c3b6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.310113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-dab04e35-dfc4-4010-83c1-980ad4c3b6dc None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:46.313098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 435/893] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:00:46.313366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dab04e35-dfc4-4010-83c1-980ad4c3b6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:46.334886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2e1e2-d274-4071-a981-231045ac7d56 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.409105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 459/894] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:46.430597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab04e35-dfc4-4010-83c1-980ad4c3b6dc None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.449034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:46.465506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2e1e2-d274-4071-a981-231045ac7d56 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:46.479266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:46.502001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:46.502957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.503325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.503666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.504386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:46.505855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:46.518587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:46.519622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:46.520477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 436/895] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:46.526810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.548691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab04e35-dfc4-4010-83c1-980ad4c3b6dc None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:46.591212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:46.592512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.593010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.593799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.607143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:46.617599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:46.618588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:00:46.619450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:46.624397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 460/896] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 07:00:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:46.625312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.656068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd260303-25ad-48e3-82f4-7194d5e2e8f8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:46.702440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:46.703704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.704137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.704717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.705989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:46.706714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:46.708263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:46.720484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 437/897] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:46.721634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.726806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:46.759427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a6b35-7b74-48f2-86b3-67ac6e47f688 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:46.774883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:46.803787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:46.805294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.805840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.806419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.807489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:46.808025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:46.809655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:46.819650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 461/898] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 07:00:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:46.825423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:46.851222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f569f5d-e4cd-4011-8573-fcf515bcfd79 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:46.898455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:46.899545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:46.900598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:46.901103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:46.902272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:46.902781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:46.904670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:46.917336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 438/899] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:46.972061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797a48b-6088-40ea-a3b1-fb102b6bdd55 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:47.018776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:47.019610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.019893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.020166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.023563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:47.033815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:47.036058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:47.055537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:47.056615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:47.057983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:47.057983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:47.057983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:47.057983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:47.060135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 462/900] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:47.066823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:47.172908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e998a-c7a9-4535-9aa9-4ae763e949a9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:47.175724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:47.210117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:47.211148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.211625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.212199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.213142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:47.213661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:47.215231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:47.227895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 439/901] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:47.233825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:47.354578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a4f63-7262-4827-b5cc-dfd15aab8e33 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:47.390377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:47.391626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.392104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.392567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.402294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:47.404171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:47.420689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:47.421742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:47.422844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:47.425541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 463/902] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:00:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:47.431398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:47.535968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d71ea-961a-498f-b684-cf9bbe58ecea None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:47.570282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86797dc1-248b-4b89-98f2-737c80edeae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:47.571320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86797dc1-248b-4b89-98f2-737c80edeae6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.571795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86797dc1-248b-4b89-98f2-737c80edeae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.572290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86797dc1-248b-4b89-98f2-737c80edeae6 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.573420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86797dc1-248b-4b89-98f2-737c80edeae6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:47.574279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86797dc1-248b-4b89-98f2-737c80edeae6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:47.575915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86797dc1-248b-4b89-98f2-737c80edeae6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:47.601398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 440/903] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 07:00:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:47.603311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:47.654679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3685e-7cfd-4f96-bc27-75e5ea035021 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.655001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3685e-7cfd-4f96-bc27-75e5ea035021 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.655235 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3685e-7cfd-4f96-bc27-75e5ea035021 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.655807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:47.660425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 441/904] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:47.661961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c609-43c1-4fbf-bc89-f1fbe09ccbda None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.662249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c609-43c1-4fbf-bc89-f1fbe09ccbda None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.662529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c609-43c1-4fbf-bc89-f1fbe09ccbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.679541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8425c609-43c1-4fbf-bc89-f1fbe09ccbda None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:47.680731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8425c609-43c1-4fbf-bc89-f1fbe09ccbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:47.699289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:47.725067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a4b59-0aa0-4ac2-b309-322229769173 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:47.761190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 442/905] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:00:47.766011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:47.766891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.767365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.767876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.776995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c609-43c1-4fbf-bc89-f1fbe09ccbda None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:47.849260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:47.850639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:47.896493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c609-43c1-4fbf-bc89-f1fbe09ccbda None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:47.899777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:47.940573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:47.941428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:47.941710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:47.941983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:47.942638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:47.944519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:47.960001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:47.960982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 07:00:47.962277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:47.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 464/906] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:00:47.969515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:47.970162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 443/907] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Jul 12 07:00:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:47.974192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.009817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:48.090354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f19c9d-866b-4ef3-baff-21e438404f89 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.117587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed04fd1-011e-4851-81d0-feadcbf1ba98 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.139310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.140709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.141689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.142173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.155228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.156718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.157426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.157711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.157982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.158197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.158691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:48.158991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.160348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.170349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 444/908] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:48.172118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:48.173241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:48.183563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:48.184505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:48.186040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:48.186040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:48.186040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:48.186040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:48.186040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 465/909] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:48.190380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.267682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:48.289583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10522-3b14-40b9-8f0c-db58bfdbd0f7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.318202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d87e73-99a1-48f1-bb0f-f180884eb65e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.326413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4c6e2-5a46-4734-9ebb-24176143d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.327085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c6e2-5a46-4734-9ebb-24176143d6fc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.327523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c6e2-5a46-4734-9ebb-24176143d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.327800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c6e2-5a46-4734-9ebb-24176143d6fc None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.328507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4c6e2-5a46-4734-9ebb-24176143d6fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:48.328808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4c6e2-5a46-4734-9ebb-24176143d6fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.330249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4c6e2-5a46-4734-9ebb-24176143d6fc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.337085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 445/910] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:48.337085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e493c-08c4-4113-9749-610541ea5f97 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.337085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e493c-08c4-4113-9749-610541ea5f97 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.337085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e493c-08c4-4113-9749-610541ea5f97 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.338201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 446/911] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:48.342593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e493c-08c4-4113-9749-610541ea5f97 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.359824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f6d2f-b522-4254-8d33-6f21c7cce27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.360497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f6d2f-b522-4254-8d33-6f21c7cce27e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.360778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f6d2f-b522-4254-8d33-6f21c7cce27e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.361091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f6d2f-b522-4254-8d33-6f21c7cce27e None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.361803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f6d2f-b522-4254-8d33-6f21c7cce27e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:48.362107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f6d2f-b522-4254-8d33-6f21c7cce27e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.363479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f6d2f-b522-4254-8d33-6f21c7cce27e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.380422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 466/912] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:48.380422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d3f37-8302-4637-b417-79ec687548b0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.380422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d3f37-8302-4637-b417-79ec687548b0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.380422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d3f37-8302-4637-b417-79ec687548b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.398552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-370d3f37-8302-4637-b417-79ec687548b0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:48.399727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-370d3f37-8302-4637-b417-79ec687548b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:48.457284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e493c-08c4-4113-9749-610541ea5f97 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.460111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:48.473271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 467/913] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:48.474351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d3f37-8302-4637-b417-79ec687548b0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.506456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.507130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.507416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.507694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.508448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.509860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.526907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:48.533533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 447/914] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:00:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:48.534537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.605642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d3f37-8302-4637-b417-79ec687548b0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.662121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.663524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.664399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.665138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.667767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ddc6d-cfb0-4c41-a3fa-6900d2912d57 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.675208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.677742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:48.697088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:48.698097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:48.699377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:48.701699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 468/915] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:48.704719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.717645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.719019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.719704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.719985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.720698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.723161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:48.736644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:48.737703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:48.738559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 448/916] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:48.745971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.769250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:48.814975 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:48.831951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254512b5-5e53-422b-b4d1-67a665ccc219 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.856375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dba0f-d375-4e89-93c3-18a93473742f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:48.875178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.875896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.876122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.876394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.877337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:48.877511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.878949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.901094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:48.901094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:48.901094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:48.901094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:48.901505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:48.901505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:48.901863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:48.902074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 469/917] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:48.903876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:48.907837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 449/918] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:48.914084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:49.028250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce421239-3e0b-49d6-be22-06f0464217ed None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:49.034573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce985b0-3884-4b4d-a1e7-e202f877cd78 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:49.037181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:49.064294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:49.065587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9787a4-fea1-4a8f-95f6-52791fab908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:49.066235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9787a4-fea1-4a8f-95f6-52791fab908d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:49.066514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9787a4-fea1-4a8f-95f6-52791fab908d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:49.066978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9787a4-fea1-4a8f-95f6-52791fab908d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:49.067720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9787a4-fea1-4a8f-95f6-52791fab908d None admin] RBAC: Adding query filter params (role.id=0e26787cf1fa43b5950d51f66216162e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=d901a025bdee47e4800f8a250bef1245) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:49.068043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9787a4-fea1-4a8f-95f6-52791fab908d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:49.069507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9787a4-fea1-4a8f-95f6-52791fab908d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:49.071888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:49.073009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:49.073326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:49.073958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:49.074660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:49.074961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:49.081230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 470/919] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/role_assignments?user.id=d901a025bdee47e4800f8a250bef1245&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=0e26787cf1fa43b5950d51f66216162e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:49.081449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:49.081546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03cd79-ad4e-4acb-bbff-68faaa42e270 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:49.081546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03cd79-ad4e-4acb-bbff-68faaa42e270 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:49.081546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03cd79-ad4e-4acb-bbff-68faaa42e270 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:49.081546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 471/920] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:49.086835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 450/921] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 07:00:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:49.090778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc68e9-7b59-4e5d-bbfd-e064633ff627 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:49.091083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc68e9-7b59-4e5d-bbfd-e064633ff627 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:49.091485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc68e9-7b59-4e5d-bbfd-e064633ff627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:49.093768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:49.110905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f9dc68e9-7b59-4e5d-bbfd-e064633ff627 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:49.112592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f9dc68e9-7b59-4e5d-bbfd-e064633ff627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:49.193081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 472/922] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:49.211505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc68e9-7b59-4e5d-bbfd-e064633ff627 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:49.213405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90cd46-afbf-4d1e-8ea4-6ebe97ae1f07 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:49.258879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f317435-7e67-4a07-974e-51b5325a4190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:49.259884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f317435-7e67-4a07-974e-51b5325a4190 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:49.260165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f317435-7e67-4a07-974e-51b5325a4190 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:49.260436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f317435-7e67-4a07-974e-51b5325a4190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:49.262114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f317435-7e67-4a07-974e-51b5325a4190 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:49.263363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f317435-7e67-4a07-974e-51b5325a4190 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:49.297470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 451/923] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:00:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:49.343279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:49.347482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc68e9-7b59-4e5d-bbfd-e064633ff627 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:49.394266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b15e10-4910-40a5-a3ee-4ccb5578e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:49.395258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b15e10-4910-40a5-a3ee-4ccb5578e644 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:49.395739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b15e10-4910-40a5-a3ee-4ccb5578e644 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:49.396178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b15e10-4910-40a5-a3ee-4ccb5578e644 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:49.397208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b15e10-4910-40a5-a3ee-4ccb5578e644 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:49.398775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b15e10-4910-40a5-a3ee-4ccb5578e644 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:49.416816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 473/924] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:00:49 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:00:50.729649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb14a-b46e-4220-8de0-dd058b92e08c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:50.730185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb14a-b46e-4220-8de0-dd058b92e08c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:50.730185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb14a-b46e-4220-8de0-dd058b92e08c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:50.731743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 452/925] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:50.737269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075cdd-671f-4f1d-883d-ff812ee584c0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:50.737661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075cdd-671f-4f1d-883d-ff812ee584c0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:50.737825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5075cdd-671f-4f1d-883d-ff812ee584c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:50.755583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a5075cdd-671f-4f1d-883d-ff812ee584c0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:50.756782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a5075cdd-671f-4f1d-883d-ff812ee584c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:50.836528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 474/926] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:50.860066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afb14a-b46e-4220-8de0-dd058b92e08c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:50.914952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:50.977036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afb14a-b46e-4220-8de0-dd058b92e08c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.012889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a19edb-8e4e-4818-ac31-a4c6196ee4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:51.013728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a19edb-8e4e-4818-ac31-a4c6196ee4de None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.014005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a19edb-8e4e-4818-ac31-a4c6196ee4de None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.014275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a19edb-8e4e-4818-ac31-a4c6196ee4de None admin] PATH_INFO: `/v3/services/compute` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.015307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a19edb-8e4e-4818-ac31-a4c6196ee4de None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:51.016686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a19edb-8e4e-4818-ac31-a4c6196ee4de None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c6a19edb-8e4e-4818-ac31-a4c6196ee4de None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:51.029186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:51.030239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.031137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 453/927] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:51.037601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075cdd-671f-4f1d-883d-ff812ee584c0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.119938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:51.138177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5075cdd-671f-4f1d-883d-ff812ee584c0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.173558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc2b6e-22ce-4110-a627-898c57ba5686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:51.174227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2b6e-22ce-4110-a627-898c57ba5686 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.174506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2b6e-22ce-4110-a627-898c57ba5686 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.174783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2b6e-22ce-4110-a627-898c57ba5686 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.175474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2b6e-22ce-4110-a627-898c57ba5686 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:51.175773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2b6e-22ce-4110-a627-898c57ba5686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:51.177165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2b6e-22ce-4110-a627-898c57ba5686 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:51.182396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 475/928] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:51.182934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46d109-310f-4bc3-8c80-68a61c8b2af1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.183295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46d109-310f-4bc3-8c80-68a61c8b2af1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.183654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46d109-310f-4bc3-8c80-68a61c8b2af1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.185240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 454/929] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:51.190478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649d1fc-c524-45db-9ea6-1aa352443695 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.190957 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649d1fc-c524-45db-9ea6-1aa352443695 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.191399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649d1fc-c524-45db-9ea6-1aa352443695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.191520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46d109-310f-4bc3-8c80-68a61c8b2af1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.208592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2649d1fc-c524-45db-9ea6-1aa352443695 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:51.210027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2649d1fc-c524-45db-9ea6-1aa352443695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:51.293149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 476/930] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:51.307654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649d1fc-c524-45db-9ea6-1aa352443695 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.321213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46d109-310f-4bc3-8c80-68a61c8b2af1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.340658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:51.368664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ec5e-c8bd-4c63-8e6c-644e96862ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:51.369865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ec5e-c8bd-4c63-8e6c-644e96862ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.370286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ec5e-c8bd-4c63-8e6c-644e96862ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.370936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ec5e-c8bd-4c63-8e6c-644e96862ec9 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.372194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ec5e-c8bd-4c63-8e6c-644e96862ec9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:51.372895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ec5e-c8bd-4c63-8e6c-644e96862ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:51.374755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ec5e-c8bd-4c63-8e6c-644e96862ec9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:51.381542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 455/931] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:51.406754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:51.426446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9f1c9-5eb0-4b1b-bb6f-63b98e47afb8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.426763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9f1c9-5eb0-4b1b-bb6f-63b98e47afb8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.427008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9f1c9-5eb0-4b1b-bb6f-63b98e47afb8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.429478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 456/932] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:51.437717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649d1fc-c524-45db-9ea6-1aa352443695 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.440153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b8b5f-e826-4c84-b92c-960393b1473f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.441335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b8b5f-e826-4c84-b92c-960393b1473f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.441633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b8b5f-e826-4c84-b92c-960393b1473f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.466428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-876b8b5f-e826-4c84-b92c-960393b1473f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:51.468348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-876b8b5f-e826-4c84-b92c-960393b1473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:51.478544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a235d-fb37-41e1-9b17-cf9e6d6bf2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:51.479394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a235d-fb37-41e1-9b17-cf9e6d6bf2ef None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.479669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a235d-fb37-41e1-9b17-cf9e6d6bf2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.479939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a235d-fb37-41e1-9b17-cf9e6d6bf2ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.491572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a235d-fb37-41e1-9b17-cf9e6d6bf2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3c3a235d-fb37-41e1-9b17-cf9e6d6bf2ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:51.500389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:51.501438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:51.502286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 477/933] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 07:00:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:51.503653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca194f-7f9d-4155-bead-d2097c6b7709 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.503756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca194f-7f9d-4155-bead-d2097c6b7709 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.504008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca194f-7f9d-4155-bead-d2097c6b7709 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.505634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 478/934] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:51.507715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca194f-7f9d-4155-bead-d2097c6b7709 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.557729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 457/935] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:00:51.559324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce62a3-a752-4293-8c22-03c17ae85cb8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.559609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce62a3-a752-4293-8c22-03c17ae85cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.559888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce62a3-a752-4293-8c22-03c17ae85cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.578805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ebce62a3-a752-4293-8c22-03c17ae85cb8 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:51.580096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ebce62a3-a752-4293-8c22-03c17ae85cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:51.626330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca194f-7f9d-4155-bead-d2097c6b7709 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.657468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 458/936] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:51.659361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce62a3-a752-4293-8c22-03c17ae85cb8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.663206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:51.664425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.664425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.664618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.665460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:51.665821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:51.667389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:51.670274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:51.677880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 479/937] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 07:00:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:51.678919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.703124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:51.776888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce62a3-a752-4293-8c22-03c17ae85cb8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.793063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf248c7-eeda-4f4e-a7ae-f03aeebf73a2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.813463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:51.814056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.814336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.814607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.815307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:51.816734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:51.818129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING keystone.server.flask.common [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 07:00:51.829336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:51.830406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:51.830865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:51.831317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:51.832216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:51.833764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:51.837238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.sql.core [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 07:00:51.837238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 07:00:51.837238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [parameters: {'id': '7fd73aab758243fb9446b4d0745fb863', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 07:00:51.837238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80472) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fd73aab758243fb9446b4d0745fb863.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fd73aab758243fb9446b4d0745fb863. Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 07:00:51.845799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 07:00:51.846762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application yield resource Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise self.value Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application session.commit() Jul 12 07:00:51.848014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.flush() Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application compat.raise_( Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 07:00:51.848996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 07:00:51.849962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 07:00:51.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [parameters: {'id': '7fd73aab758243fb9446b4d0745fb863', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.851667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:51.852615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fd73aab758243fb9446b4d0745fb863. Jul 12 07:00:51.853586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:51.854102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:51.856976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:51.858036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 480/938] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:51.859297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.865115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 459/939] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Jul 12 07:00:51 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:51.865115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:51.918688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:51.962142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:51.972089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e9a7b-1df7-42b0-9042-c7b02863868f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:51.973625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aad27-f229-40e3-93ca-5b6ba416fa45 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.012438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.013168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.013455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.013731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.014436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:52.014846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.016419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.020599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.021683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.022159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.022612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.028647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 460/940] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:52.028647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.035696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.037264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:52.055058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:52.056080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:52.057777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:52.057777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:52.057777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:52.057777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:52.057777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 481/941] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:52.057777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.138037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857a9ec-fefc-4ee8-8ada-42ca5f6c7c42 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.170214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8170d9-d9a8-488e-aa5b-71c81b30960e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.172228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.173036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.173375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.173625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.181707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.183472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.186857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:52.199177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:52.200290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 461/942] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:52.201242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.215832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.217068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.217521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.218341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.219207 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:52.219665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.221294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.224477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:52.227464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 482/943] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:52.228539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.309115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b75b08-6aa8-4be8-9b5a-6e92359bf0fc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.337445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b18409-5685-43e0-99b9-a98f7b3399be None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.345150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.345819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.346094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.346361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.347093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:52.347392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.348765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.364772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 462/944] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:52.364772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.377699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.378352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.378647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.378918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.379694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:52.380087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.381664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.400156 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 483/945] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:52.400912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.452078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:52.458299 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:52.483592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c860f1d-0710-404f-ba4c-a641c099c9ff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.522663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.523760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.524279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.524771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.525007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d786-b47f-4143-94d9-3062f55dab78 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.526102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] RBAC: Adding query filter params (interface=public, service_id=9ba4b53cab304509812b045c53d2d7cb, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:52.526657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.528304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.539905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 463/946] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/endpoints?service_id=9ba4b53cab304509812b045c53d2d7cb&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:52.541285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.566256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.566592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.566838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.567111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.575415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.576875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:52.595051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:52.596096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:52.597580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:52.601101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 484/947] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:52.602352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.670207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20fdf8-7708-4074-b7b7-7d1df42ef74d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.713158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.713836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.714183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.714389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.715245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.716385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:52.716933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.722040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0426cb-af84-4c4a-8965-0f6e1c82522e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.729091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 464/948] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:00:52.762959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.763667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.763940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.764218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.765226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:52.765540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.767009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.792002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 485/949] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:52.793514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:52.798056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:52.912924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170013c-2670-4fca-9f7d-ec1924b40033 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:52.948123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:52.949316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:52.949628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:52.949869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:52.961789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:52.963264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:52.981406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:52.982349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:52.983256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:52.983256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:52.983256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:52.983256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:52.987921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 465/950] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:52.989413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:53.095183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17854a9-a0d7-460e-8d9a-89e404340f14 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:53.130784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:53.131558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:53.131840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:53.132118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:53.132846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:53.133185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:53.134657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:53.151536 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 486/951] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:53.153127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:53.212894 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:53.261574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2df60e-b666-4449-bfd9-f312a5f03760 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:53.295049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e62405-bf29-45ad-9a99-a180005671ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:53.296378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e62405-bf29-45ad-9a99-a180005671ef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:53.296859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e62405-bf29-45ad-9a99-a180005671ef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:53.297391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e62405-bf29-45ad-9a99-a180005671ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:53.304392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e62405-bf29-45ad-9a99-a180005671ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:53.306134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e62405-bf29-45ad-9a99-a180005671ef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b9e62405-bf29-45ad-9a99-a180005671ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:53.320791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:53.321786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:53.322687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:53.325732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 466/952] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:53.333080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:53.352238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:53.435656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf819-6e93-4839-ba98-8434debcae10 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:53.462846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15497-2033-47ea-828a-9c0f97496663 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:53.463190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15497-2033-47ea-828a-9c0f97496663 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:53.463406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15497-2033-47ea-828a-9c0f97496663 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:53.464811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 467/953] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:53.471278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434891c6-caeb-4c31-a95c-9d517d02fd83 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:53.471659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434891c6-caeb-4c31-a95c-9d517d02fd83 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:53.472010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434891c6-caeb-4c31-a95c-9d517d02fd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:53.475280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:53.476135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:53.476560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:53.476976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:53.478003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:53.478488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:53.479967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:53.491328 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-434891c6-caeb-4c31-a95c-9d517d02fd83 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:53.493190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-434891c6-caeb-4c31-a95c-9d517d02fd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:53.497132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 487/954] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:53.502076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:53.576334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 468/955] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:00:53.595471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434891c6-caeb-4c31-a95c-9d517d02fd83 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:53.609460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:53.637359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e804d0-22db-45f3-be74-cb39da34884e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:53.649106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:53.679519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce40d3-4a26-4184-80e3-a5a672ff9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:53.680680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce40d3-4a26-4184-80e3-a5a672ff9536 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:53.681231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce40d3-4a26-4184-80e3-a5a672ff9536 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:53.681704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce40d3-4a26-4184-80e3-a5a672ff9536 None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/d901a025bdee47e4800f8a250bef1245/roles/0e26787cf1fa43b5950d51f66216162e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:53.684203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce40d3-4a26-4184-80e3-a5a672ff9536 None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=d901a025bdee47e4800f8a250bef1245, role_id=0e26787cf1fa43b5950d51f66216162e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:53.685911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce40d3-4a26-4184-80e3-a5a672ff9536 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:53.703189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 488/956] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:00:53 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/d901a025bdee47e4800f8a250bef1245/roles/0e26787cf1fa43b5950d51f66216162e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:00:53.716752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434891c6-caeb-4c31-a95c-9d517d02fd83 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:53.758503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af56a0-7396-405a-bd27-0ff5095a42fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:53.759576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af56a0-7396-405a-bd27-0ff5095a42fa None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:53.760081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af56a0-7396-405a-bd27-0ff5095a42fa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:53.760537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af56a0-7396-405a-bd27-0ff5095a42fa None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:53.761488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af56a0-7396-405a-bd27-0ff5095a42fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:53.763167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af56a0-7396-405a-bd27-0ff5095a42fa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:53.783571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 469/957] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:00:53 2022] POST /identity/v3/services => generated 242 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:54.580288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f2c0-e07a-451d-9ce2-c2a6c3bc95b4 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:54.580852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f2c0-e07a-451d-9ce2-c2a6c3bc95b4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:54.580852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f2c0-e07a-451d-9ce2-c2a6c3bc95b4 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:54.582510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 489/958] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:54.587718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789a3ab-545c-49c9-b09d-ce8c4badcb77 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:54.588119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789a3ab-545c-49c9-b09d-ce8c4badcb77 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:54.588284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789a3ab-545c-49c9-b09d-ce8c4badcb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:54.605675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1789a3ab-545c-49c9-b09d-ce8c4badcb77 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:54.607039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1789a3ab-545c-49c9-b09d-ce8c4badcb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:54.682683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 470/959] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:54.707661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1f2c0-e07a-451d-9ce2-c2a6c3bc95b4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:54.813202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1f2c0-e07a-451d-9ce2-c2a6c3bc95b4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:54.849035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:54.849661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:54.849943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:54.850216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:54.850928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:54.852374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:54.853720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING keystone.server.flask.common [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 07:00:54.857253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3362c-e7df-48f0-a7ae-610dc061f12f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:54.857545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3362c-e7df-48f0-a7ae-610dc061f12f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:54.857823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3362c-e7df-48f0-a7ae-610dc061f12f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:54.859603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 471/960] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:54.862288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:54.864800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e24b9-9e06-42d1-be15-8faeb84614b6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:54.865159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e24b9-9e06-42d1-be15-8faeb84614b6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:54.865440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e24b9-9e06-42d1-be15-8faeb84614b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:54.868701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.sql.core [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 07:00:54.868701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 07:00:54.868701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [parameters: {'user_id': '4a028f671304494e8d55a249cc924f1a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 07:00:54.868701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80473) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 07:00:54.885114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bd0e24b9-9e06-42d1-be15-8faeb84614b6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:54.886705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bd0e24b9-9e06-42d1-be15-8faeb84614b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application conn.query(q) Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result.read() Jul 12 07:00:54.888308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 07:00:54.889338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application next(self.gen) Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application yield resource Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application next(self.gen) Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise self.value Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application session.commit() Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 07:00:54.890635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.session.flush() Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application compat.raise_( Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 07:00:54.891812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 07:00:54.900899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application conn.query(q) Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application result.read() Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 07:00:54.903369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [parameters: {'user_id': '4a028f671304494e8d55a249cc924f1a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:54.905527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 07:00:54.909387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 490/961] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 07:00:54 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:54.911574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:54.937093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:55.002460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 472/962] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:55.028083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e24b9-9e06-42d1-be15-8faeb84614b6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:55.033190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d1422-0e3f-471d-90db-24ad931d74da None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:55.072107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:55.073332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.074201 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.074680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.083891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:55.086034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:55.103280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:55.104273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 07:00:55.105322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:55.108234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 491/963] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:00:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:55.115495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:55.146503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e24b9-9e06-42d1-be15-8faeb84614b6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:55.182373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:55.183209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.183496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.183763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.184411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:55.185844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:55.191816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:55.198368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:55.199799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:55.200702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:55.205001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 473/964] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:55.205981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:55.235130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:55.244382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8fee-3d8c-4980-8d49-1b4665ac1b3f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:55.281883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:55.282927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.283458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.283924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.284930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:55.285453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:55.287112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:55.337364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788934eb-b5da-4468-a523-542ed164c8d8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:55.337764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 492/965] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 07:00:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:55.373086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:55.373704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.373989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.374468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.375201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:55.375560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:55.377030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:55.383382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 474/966] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:55.389492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:55.503321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8f60b-16cb-4e83-8f3b-24c48a721bca None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:55.523054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:55.545799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b5a83-ccfc-4f5d-b05b-ca09cb58628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:55.546913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b5a83-ccfc-4f5d-b05b-ca09cb58628e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.547214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b5a83-ccfc-4f5d-b05b-ca09cb58628e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.547474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b5a83-ccfc-4f5d-b05b-ca09cb58628e None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.548189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b5a83-ccfc-4f5d-b05b-ca09cb58628e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:55.548497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b5a83-ccfc-4f5d-b05b-ca09cb58628e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:55.549930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b5a83-ccfc-4f5d-b05b-ca09cb58628e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:55.558077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 493/967] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:55.563999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:55.667748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:55.678667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78619256-e02e-4cd9-b149-1cd09ac804de None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:55.715607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020abf3-3560-4e33-ad8e-d35acf1adcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:55.716282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020abf3-3560-4e33-ad8e-d35acf1adcbd None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.716553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020abf3-3560-4e33-ad8e-d35acf1adcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.716827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020abf3-3560-4e33-ad8e-d35acf1adcbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.717595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020abf3-3560-4e33-ad8e-d35acf1adcbd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:55.719105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020abf3-3560-4e33-ad8e-d35acf1adcbd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:55.738433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 475/968] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:00:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:00:55.809327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9378d-1a20-4250-8813-7ea677d11b55 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.809863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9378d-1a20-4250-8813-7ea677d11b55 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.809863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9378d-1a20-4250-8813-7ea677d11b55 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.812356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 494/969] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:55.818271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db039bf-2467-4fb9-8768-02578a5a7e27 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.818627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db039bf-2467-4fb9-8768-02578a5a7e27 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.818790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db039bf-2467-4fb9-8768-02578a5a7e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.838271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3db039bf-2467-4fb9-8768-02578a5a7e27 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:55.839532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3db039bf-2467-4fb9-8768-02578a5a7e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:55.922269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 476/970] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:55.924930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920111a2-9a49-4082-87de-25dd8d8b1c39 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.925255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920111a2-9a49-4082-87de-25dd8d8b1c39 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.925531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920111a2-9a49-4082-87de-25dd8d8b1c39 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.926911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 495/971] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:55.932129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58fbea-df3c-4744-8b4b-c47668bbcd77 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:55.932418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58fbea-df3c-4744-8b4b-c47668bbcd77 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:55.932694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58fbea-df3c-4744-8b4b-c47668bbcd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:55.945120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920111a2-9a49-4082-87de-25dd8d8b1c39 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:55.950322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-db58fbea-df3c-4744-8b4b-c47668bbcd77 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:55.951512 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db58fbea-df3c-4744-8b4b-c47668bbcd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:56.045091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 477/972] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:56.054005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920111a2-9a49-4082-87de-25dd8d8b1c39 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.058809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58fbea-df3c-4744-8b4b-c47668bbcd77 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.061213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:56.091600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.092456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.092741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.093197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.105128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:00:56.114776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:56.115778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:56.116641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 496/973] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 07:00:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:56.122102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.164534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:56.176813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58fbea-df3c-4744-8b4b-c47668bbcd77 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.213244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.214051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.214360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.214654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.215288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.216657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.227240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85c48e-3635-4568-911a-4c9f2e3391bf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:56.228864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:56.229921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:56.230909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 478/974] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:56.235989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.264940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.265637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.265920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.266193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.266942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:56.267243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.268619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.277107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 497/975] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:56.283137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.310650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:56.350587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab3b0-8efb-438f-966d-2ec7dc09ed4f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.387078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.387745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.388022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.388297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.388461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b46445-4592-424a-a61d-4a18bc31dc2c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.388998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:56.389323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.390179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:56.392157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.398260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 479/976] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:56.403595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.426061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.426970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.427154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.427421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.439067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.440521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:56.458466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:56.459453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:56.460381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:56.460381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:56.460381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:56.460381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:56.460381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 498/977] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:56.465304 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.514104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69789b75-645c-4448-ab3a-e588d08fea36 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.551617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:56.558255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.559075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.559461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.560234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.560650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:56.561358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.563478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.574583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 480/978] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:56.575674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.579386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd98eec-7bc9-4f2e-8493-25a1c0bbdf8e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.621527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.622254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.622537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.622812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.623540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:56.623943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.625500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.638182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 499/979] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:56.643793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.664277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:56.701783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cfe19-5c66-49c0-a992-9ed2ebe4a75f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.739351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.740023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.740305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.740581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.741674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] RBAC: Adding query filter params (interface=public, service_id=2c4914a6b4344c83b263c04a57167e9a, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:56.741953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.743386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.750477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 481/980] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/endpoints?service_id=2c4914a6b4344c83b263c04a57167e9a&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:56.756290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.771359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf659aa5-0264-4b62-9e38-e1539ed2be05 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.807415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.808389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.808677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.809120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.817062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.817991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:56.819214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:00:56.836599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:56.838315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:56.839300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 500/981] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:00:56.843368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:56.876011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfefe5-f225-4604-8c89-342f55e26615 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:56.913022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:56.913726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:56.914010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:56.914284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:56.915137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:56.916341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:56.923433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 482/982] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:00:57.004728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:57.034723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ac9cc-c73e-410b-83e2-b8cea968e535 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:57.072443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:57.075991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:57.075991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:57.075991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:57.075991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:57.075991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:57.078747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:57.098667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 501/983] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 07:00:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:57.105792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:57.217301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37abae-70dd-412b-bfc8-ce2a6269ffd1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:57.242321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:57.255817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:57.256482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:57.256764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:57.257042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:57.330798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:57.332264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:57.436437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 483/984] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:00:57 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:00:57.442772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:57.548070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711f374-5de3-4713-b5ae-75854807cac7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:57.579350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:57.586210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:57.587093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:57.587433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:57.587764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:57.598579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:57.600107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:00:57.622498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:57.623637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:57.624868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:00:57.624868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:00:57.624868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:00:57.624868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:57.629405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 502/985] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:57.630688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:57.655791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:57.737954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca454dc-8cc5-409c-83a7-8e6503f2dea4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:57.772111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbbc9b-89a9-4bce-8c30-148d72827635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:57.772775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbbc9b-89a9-4bce-8c30-148d72827635 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:57.773077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbbc9b-89a9-4bce-8c30-148d72827635 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:57.773350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbbc9b-89a9-4bce-8c30-148d72827635 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:57.774064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbbc9b-89a9-4bce-8c30-148d72827635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:57.774361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbbc9b-89a9-4bce-8c30-148d72827635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:57.775720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbbc9b-89a9-4bce-8c30-148d72827635 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:57.787508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 484/986] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:57.792929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:57.815739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177acf2-9ea0-46d7-9672-5891f6fee9a1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:57.815916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177acf2-9ea0-46d7-9672-5891f6fee9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:57.816218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177acf2-9ea0-46d7-9672-5891f6fee9a1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:57.817869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 485/987] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:00:57.822613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63263bd-ec21-482e-aef5-e0c448b2e525 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:57.822918 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63263bd-ec21-482e-aef5-e0c448b2e525 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:57.823183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63263bd-ec21-482e-aef5-e0c448b2e525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:57.841422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d63263bd-ec21-482e-aef5-e0c448b2e525 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:57.842607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d63263bd-ec21-482e-aef5-e0c448b2e525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:57.898849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f3343-202e-4b1d-860b-c2260e11b4e4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:57.915263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 486/988] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:00:57.936359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:57.937861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:57.938103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63263bd-ec21-482e-aef5-e0c448b2e525 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:57.938221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:57.938422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:57.945486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:57.947089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:57.953015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:00:57.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:57.963072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:57.963969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 503/989] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:57.969190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:58.026697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:58.052857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63263bd-ec21-482e-aef5-e0c448b2e525 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:58.089313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aced41b-2522-44cc-bde4-7c847de5bdf8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:58.093772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:58.094823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:58.095327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:58.095812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:58.096652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:58.098668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:58.111943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:58.113184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:00:58.114047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:00:58.115380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 487/990] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:00:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:58.120930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:58.132471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:58.133280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:58.133635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:58.133919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:58.134722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:58.135032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:58.136467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:58.149869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 504/991] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:58.155640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:58.202937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:58.231524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e26ea-a119-47ad-bbd8-7549604b42cd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:58.259775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212e34b-668c-4219-bb48-e547890e0f3e None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:58.262826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:58.267305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:58.267976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:58.268261 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:58.268537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:58.269271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:58.269575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:58.270948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:58.275924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 488/992] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:58.281501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:58.297224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:58.297893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:58.298175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:58.298452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:58.299287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] RBAC: Adding query filter params (role.id=0e26787cf1fa43b5950d51f66216162e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=d901a025bdee47e4800f8a250bef1245) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:58.299601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:58.301138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:58.307485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 505/993] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:00:58 2022] GET /identity/v3/role_assignments?user.id=d901a025bdee47e4800f8a250bef1245&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=0e26787cf1fa43b5950d51f66216162e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:58.395404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26863cdb-dbf5-4d74-a9f1-a4574cc11ef5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:58.433431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:58.434128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:58.434399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:58.434662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:58.435373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:58.435674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:58.437139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:58.443057 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 489/994] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:00:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:00:58.450158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:58.559531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87590d0-8b5b-4280-bf10-686328f49716 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:58.595718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459d773-e556-4186-aa90-6e025b9f8eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:58.596378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459d773-e556-4186-aa90-6e025b9f8eba None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:58.596657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459d773-e556-4186-aa90-6e025b9f8eba None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:58.596929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459d773-e556-4186-aa90-6e025b9f8eba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:58.597754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459d773-e556-4186-aa90-6e025b9f8eba None admin] RBAC: Adding query filter params (interface=admin, service_id=9ba4b53cab304509812b045c53d2d7cb, region_id=RegionOne) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:58.598059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459d773-e556-4186-aa90-6e025b9f8eba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:58.599433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459d773-e556-4186-aa90-6e025b9f8eba None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:58.605243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 506/995] 10.222.0.25 () {56 vars in 1249 bytes} [Tue Jul 12 07:00:58 2022] GET /identity/v3/endpoints?service_id=9ba4b53cab304509812b045c53d2d7cb&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:58.611057 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:58.615904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:58.718926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92365f-3eab-4eee-a4c7-29725078cabe None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:58.758959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5fb45-ccd9-480b-9bf6-1db5ec4c3d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:58.759642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5fb45-ccd9-480b-9bf6-1db5ec4c3d5f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:58.759908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5fb45-ccd9-480b-9bf6-1db5ec4c3d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:58.760161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5fb45-ccd9-480b-9bf6-1db5ec4c3d5f None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:58.760838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5fb45-ccd9-480b-9bf6-1db5ec4c3d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:58.762234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5fb45-ccd9-480b-9bf6-1db5ec4c3d5f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:58.768968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 490/996] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:00:58 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:00:59.024441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bff6d-48c6-45a6-a6c0-b6285570845f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:59.025027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bff6d-48c6-45a6-a6c0-b6285570845f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:59.025027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bff6d-48c6-45a6-a6c0-b6285570845f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:59.026371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 507/997] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:00:59.032223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477188f-3df6-486c-82df-dbb3e8af372d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:59.032513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477188f-3df6-486c-82df-dbb3e8af372d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:59.032793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477188f-3df6-486c-82df-dbb3e8af372d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:59.050290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7477188f-3df6-486c-82df-dbb3e8af372d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:00:59.051482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7477188f-3df6-486c-82df-dbb3e8af372d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:00:59.120358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:59.133209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 491/998] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:00:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:00:59.149023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bff6d-48c6-45a6-a6c0-b6285570845f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:59.188645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:59.263632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bff6d-48c6-45a6-a6c0-b6285570845f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:59.298553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:59.299353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:59.299629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:59.299901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:59.300549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:59.301935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:00:59.313913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:00:59.315027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 07:00:59.315956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:00:59.320095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 508/999] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:00:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:00:59.321100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477188f-3df6-486c-82df-dbb3e8af372d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:59.419601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477188f-3df6-486c-82df-dbb3e8af372d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:59.455091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:59.455867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:59.456042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:59.456327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:59.457076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] RBAC: Adding query filter params (name=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:59.457397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:59.458795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:59.464525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 492/1000] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:59.470582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:59.571722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:59.579476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40943d5-e9b2-43d6-b794-dea66dc55c0b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:59.613752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b08b9-2f4d-41ae-975d-9f2490a04e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:59.614573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b08b9-2f4d-41ae-975d-9f2490a04e4c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:59.614951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b08b9-2f4d-41ae-975d-9f2490a04e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:59.615315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b08b9-2f4d-41ae-975d-9f2490a04e4c None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:59.616114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b08b9-2f4d-41ae-975d-9f2490a04e4c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:00:59.616507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b08b9-2f4d-41ae-975d-9f2490a04e4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:59.617992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b08b9-2f4d-41ae-975d-9f2490a04e4c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:59.623059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 509/1001] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 07:00:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:00:59.628706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:00:59.691739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:00:59.736422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bb063-58d7-47c7-8b1c-ba986bb135ac None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:00:59.771651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f46c3-c4b5-48ad-961a-288cbf9b16e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:00:59.772318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f46c3-c4b5-48ad-961a-288cbf9b16e0 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:00:59.772579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f46c3-c4b5-48ad-961a-288cbf9b16e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:00:59.772850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f46c3-c4b5-48ad-961a-288cbf9b16e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:00:59.773607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f46c3-c4b5-48ad-961a-288cbf9b16e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:00:59.774977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f46c3-c4b5-48ad-961a-288cbf9b16e0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:00:59.799260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 493/1002] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:00:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:00.374944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff44a2-d79d-4e6a-b266-95a60ccabc96 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:00.375484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff44a2-d79d-4e6a-b266-95a60ccabc96 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:00.375484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff44a2-d79d-4e6a-b266-95a60ccabc96 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:00.376949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 510/1003] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:00.382293 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0227a9-8474-419e-8f24-5a53de0e40cb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:00.382561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0227a9-8474-419e-8f24-5a53de0e40cb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:00.382859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0227a9-8474-419e-8f24-5a53de0e40cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:00.400621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fc0227a9-8474-419e-8f24-5a53de0e40cb None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:00.401845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc0227a9-8474-419e-8f24-5a53de0e40cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:00.478335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 494/1004] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:00.500750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff44a2-d79d-4e6a-b266-95a60ccabc96 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:00.616030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff44a2-d79d-4e6a-b266-95a60ccabc96 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:00.653844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0dc6f-3427-4b20-a39d-1a6158ef5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:00.654555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0dc6f-3427-4b20-a39d-1a6158ef5aee None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:00.654833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0dc6f-3427-4b20-a39d-1a6158ef5aee None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:00.655100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0dc6f-3427-4b20-a39d-1a6158ef5aee None admin] PATH_INFO: `/v3/services/image` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:00.655933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0dc6f-3427-4b20-a39d-1a6158ef5aee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:00.657183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0dc6f-3427-4b20-a39d-1a6158ef5aee None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-4af0dc6f-3427-4b20-a39d-1a6158ef5aee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:00.668896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:00.669987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:00.670855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 511/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Jul 12 07:01:00 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:00.676488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0227a9-8474-419e-8f24-5a53de0e40cb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:00.724459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:00.781619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0227a9-8474-419e-8f24-5a53de0e40cb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:00.814520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663428d-0b52-4ce5-9ad9-2fad4ae21542 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:00.814768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663428d-0b52-4ce5-9ad9-2fad4ae21542 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:00.815066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663428d-0b52-4ce5-9ad9-2fad4ae21542 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:00.816613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 512/1006] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:01:00.816902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:00.817585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:00.817874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:00.818146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:00.818861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] RBAC: Adding query filter params (name=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:00.819168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:00.820547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:00.821347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059808-384b-4481-8aa6-9e979279a15f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:00.821617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059808-384b-4481-8aa6-9e979279a15f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:00.821892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059808-384b-4481-8aa6-9e979279a15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:00.831089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 495/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:00.832492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:00.839464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9a059808-384b-4481-8aa6-9e979279a15f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:00.840956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9a059808-384b-4481-8aa6-9e979279a15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:00.858631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:00.913706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 513/1008] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:00.936287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059808-384b-4481-8aa6-9e979279a15f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:00.936462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2410d6e-23aa-49ed-b061-87b351ceed03 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:00.948205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:00.976035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:00.976710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:00.977117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:00.977397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:00.978116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] RBAC: Adding query filter params (type=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:00.978422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:00.979808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:00.984470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 496/1009] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:01.059579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059808-384b-4481-8aa6-9e979279a15f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:01.099096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:01.099926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:01.100205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:01.100477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:01.101172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:01.102643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:01.118334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:01.119491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:01.120541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 514/1010] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 07:01:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:01.126312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:01.230215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365033-a374-46cc-8da0-8536d7825730 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:01.263354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:01.264013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:01.264291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:01.264557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:01.265343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:01.265638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:01.266998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:01.272049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 497/1011] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:01:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:01.277566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:01.298218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:01.386286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aba73d-74d7-4930-aa11-28d153992340 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:01.420247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e9d86-5798-4295-aea4-002f9eaf8f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:01.421029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e9d86-5798-4295-aea4-002f9eaf8f6c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:01.421487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e9d86-5798-4295-aea4-002f9eaf8f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:01.421768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e9d86-5798-4295-aea4-002f9eaf8f6c None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:01.422474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e9d86-5798-4295-aea4-002f9eaf8f6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:01.422783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e9d86-5798-4295-aea4-002f9eaf8f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:01.424165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e9d86-5798-4295-aea4-002f9eaf8f6c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:01.429683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 515/1012] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 07:01:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:01.435483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:01.474567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:01.542606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd416ba0-c4a2-4893-9d3e-67052b1f4d41 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:01.576957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795df90d-ea13-4a15-ac61-46dd58233882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:01.577785 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795df90d-ea13-4a15-ac61-46dd58233882 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:01.578060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795df90d-ea13-4a15-ac61-46dd58233882 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:01.578329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795df90d-ea13-4a15-ac61-46dd58233882 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:01.579059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795df90d-ea13-4a15-ac61-46dd58233882 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:01.580421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795df90d-ea13-4a15-ac61-46dd58233882 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:01.605094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 498/1013] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:01:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:01.964078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a8cff-4806-4fe3-8696-6c9025b30292 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:01.964330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a8cff-4806-4fe3-8696-6c9025b30292 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:01.964619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a8cff-4806-4fe3-8696-6c9025b30292 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:01.966220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 516/1014] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:01.971529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa29bf-168f-4af0-8b2f-592ca1965faf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:01.972003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa29bf-168f-4af0-8b2f-592ca1965faf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:01.972083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa29bf-168f-4af0-8b2f-592ca1965faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:01.991172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-76aa29bf-168f-4af0-8b2f-592ca1965faf None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:01.992693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-76aa29bf-168f-4af0-8b2f-592ca1965faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:02.074068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 499/1015] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:01 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:02.097150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a8cff-4806-4fe3-8696-6c9025b30292 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:02.183819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:02.204858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a8cff-4806-4fe3-8696-6c9025b30292 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:02.238651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:02.239389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:02.239669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:02.239939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:02.250311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:02.258354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:02.259296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:01:02.260182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:02.264455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 517/1016] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 07:01:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:02.265883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa29bf-168f-4af0-8b2f-592ca1965faf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:02.291713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:02.372189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa29bf-168f-4af0-8b2f-592ca1965faf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:02.406245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:02.406942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:02.407222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:02.407500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:02.408243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:02.408549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:02.409943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:02.422576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 500/1017] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 07:01:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:02.423812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:02.523564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0085c4-7ed4-4fc7-a530-2531375ec35c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:02.562260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:02.563063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:02.563342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:02.563617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:02.574828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:02.576284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:02.578185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:02.595291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:02.596249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:02.597208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:01:02.597208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:02.597208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:02.597208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:02.598113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 518/1018] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:02.602200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:02.704515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:02.729391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354f2af-fc39-4c2b-8973-8a3553ed4e16 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:02.767628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:02.768555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:02.769100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:02.769514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:02.770388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:02.770860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:02.772386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:02.785768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 501/1019] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:02.791045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:02.899967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96b148-e05e-4c94-906e-f93f08a11c0d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:02.934819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940e773-a8aa-4359-b751-964feb5d8643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:02.935619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940e773-a8aa-4359-b751-964feb5d8643 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:02.935906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940e773-a8aa-4359-b751-964feb5d8643 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:02.936177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940e773-a8aa-4359-b751-964feb5d8643 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:02.943873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940e773-a8aa-4359-b751-964feb5d8643 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:02.945355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940e773-a8aa-4359-b751-964feb5d8643 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-7940e773-a8aa-4359-b751-964feb5d8643 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:01:02.959704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:02.960676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:01:02.962049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:02.965373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 519/1020] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:01:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:02.966754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.020545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6411a-3733-4ff6-a72b-1cec903d13be None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.020802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6411a-3733-4ff6-a72b-1cec903d13be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.021078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6411a-3733-4ff6-a72b-1cec903d13be None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.022761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 520/1021] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:01:03.027325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6bdf-1483-4721-b276-3307644b610b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.027607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6bdf-1483-4721-b276-3307644b610b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.027904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6bdf-1483-4721-b276-3307644b610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.046802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cddf6bdf-1483-4721-b276-3307644b610b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:03.047895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cddf6bdf-1483-4721-b276-3307644b610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:03.083426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8287c75-0541-4d22-b806-9f3aa1136766 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:03.101557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:03.122358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 521/1022] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:03.123325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:03.124092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.124378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.124651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.125526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:03.125834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:03.127259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:03.145222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 502/1023] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:01:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:03.145656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6bdf-1483-4721-b276-3307644b610b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.150648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.153553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:03.248397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6bdf-1483-4721-b276-3307644b610b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:03.259144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25efe12-3137-42c8-9eee-33b3bac68227 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:03.285723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:03.286394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.286638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.287106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.287646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:03.289037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:03.295420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61121894-d3fe-4fcb-ae4c-6a914f50c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:03.296246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61121894-d3fe-4fcb-ae4c-6a914f50c146 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.297038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61121894-d3fe-4fcb-ae4c-6a914f50c146 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.297038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61121894-d3fe-4fcb-ae4c-6a914f50c146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.304940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 522/1024] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:01:03 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:01:03.379465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61121894-d3fe-4fcb-ae4c-6a914f50c146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:03.381041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61121894-d3fe-4fcb-ae4c-6a914f50c146 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:03.383475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:03.492548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 503/1025] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:03 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 07:01:03.498413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.547974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:03.614109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54b9cd-a226-4512-b0fe-ff991fe6fc5a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:03.649609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:03.650418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.650697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.651024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.659764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2769ff-532a-4b38-82ce-6390200324a6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.660048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2769ff-532a-4b38-82ce-6390200324a6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.660341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2769ff-532a-4b38-82ce-6390200324a6 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.661900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 504/1026] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:01:03.662680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:03.664310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:03.666883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a84f8-985b-4a0a-950e-865e24e47fb1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.667183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a84f8-985b-4a0a-950e-865e24e47fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.667465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a84f8-985b-4a0a-950e-865e24e47fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:03.683692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:03.684669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:03.685637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:01:03.685637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:03.685637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:03.685637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:03.685876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-348a84f8-985b-4a0a-950e-865e24e47fb1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:03.685876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-348a84f8-985b-4a0a-950e-865e24e47fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:03.689190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 523/1027] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:03.692142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.759104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 505/1028] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:03.782752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a84f8-985b-4a0a-950e-865e24e47fb1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.798530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:03.808881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d4e5d-6ad3-4eaf-9c7f-ed4ab96f5519 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:03.817996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:03.849804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:03.850474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.850753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.851036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.851742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:03.852043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:03.853444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:03.865596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 524/1029] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:03.870873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.894882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a84f8-985b-4a0a-950e-865e24e47fb1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:03.933019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:03.934101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:03.934529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:03.934950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:03.936251 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:03.938419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:03.940810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 506/1030] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:03.946247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:03.984406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117aa820-f044-4b51-8beb-b11336c4fc20 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:04.024006 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:04.024810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:04.025112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:04.025385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:04.032510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:04.034040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:01:04.048666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:04.049645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:04.050648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 525/1031] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:04.052678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:04.055738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c370382-f2fb-49c1-97dd-9eb7960fcc0f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:04.056659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:04.071472 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:04.093030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:04.093798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:04.094087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:04.094360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:04.095658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:04.096500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:04.126256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 507/1032] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:01:03 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:01:04.169811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb26ef6-801c-44b6-93b2-b8889bd5123c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:04.209418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab030cc4-bbb8-4e4c-86f7-ba050edc8d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:04.210105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030cc4-bbb8-4e4c-86f7-ba050edc8d69 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:04.210608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030cc4-bbb8-4e4c-86f7-ba050edc8d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:04.210884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030cc4-bbb8-4e4c-86f7-ba050edc8d69 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:04.211688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab030cc4-bbb8-4e4c-86f7-ba050edc8d69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:04.211995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab030cc4-bbb8-4e4c-86f7-ba050edc8d69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:04.213513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab030cc4-bbb8-4e4c-86f7-ba050edc8d69 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:04.231464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 526/1033] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:04.232861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:04.347193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077be5a-a34a-4dfe-8cb1-08a052950ada None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:04.381405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537a801-36c5-4c9b-9f87-388b64b91691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:04.382068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537a801-36c5-4c9b-9f87-388b64b91691 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:04.382398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537a801-36c5-4c9b-9f87-388b64b91691 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:04.382662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537a801-36c5-4c9b-9f87-388b64b91691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:04.383488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537a801-36c5-4c9b-9f87-388b64b91691 None admin] RBAC: Adding query filter params (role.id=0e26787cf1fa43b5950d51f66216162e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=95c860dcbf3b40bbb7427a417357c68a) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:04.383796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537a801-36c5-4c9b-9f87-388b64b91691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:04.385322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537a801-36c5-4c9b-9f87-388b64b91691 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:04.391341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 508/1034] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:04 2022] GET /identity/v3/role_assignments?user.id=95c860dcbf3b40bbb7427a417357c68a&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=0e26787cf1fa43b5950d51f66216162e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:05.378342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e98abd-0543-45e6-8aee-e694c339560a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:05.378880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e98abd-0543-45e6-8aee-e694c339560a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:05.378880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e98abd-0543-45e6-8aee-e694c339560a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:05.381091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 527/1035] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:05.385416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5960-91fa-4cc3-9372-c52e52ad5056 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:05.385813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5960-91fa-4cc3-9372-c52e52ad5056 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:05.385991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5960-91fa-4cc3-9372-c52e52ad5056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:05.403273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3edb5960-91fa-4cc3-9372-c52e52ad5056 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:05.404464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3edb5960-91fa-4cc3-9372-c52e52ad5056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:05.416567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:05.479876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 509/1036] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:05.502483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e98abd-0543-45e6-8aee-e694c339560a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:05.577749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:05.610754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e98abd-0543-45e6-8aee-e694c339560a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:05.645810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:05.646520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:05.646798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:05.647065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:05.647702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:05.649105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:05.660589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:05.661626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:05.662516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:05.665141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 528/1037] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Jul 12 07:01:05 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:05.667923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5960-91fa-4cc3-9372-c52e52ad5056 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:05.769584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5960-91fa-4cc3-9372-c52e52ad5056 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:05.803010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:05.803683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:05.803963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:05.804230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:05.804939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] RBAC: Adding query filter params (name=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:05.805262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:05.806623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:05.811828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 510/1038] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:05.817330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:05.917164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f11e8-610d-4ddd-96a4-cbae286940c7 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:05.951082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:05.951750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:05.952067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:05.952398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:05.953127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] RBAC: Adding query filter params (type=image) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:05.953432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:05.954802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:05.956921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:05.965220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 529/1039] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:05.966932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:05.979676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:06.078445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf4777-7fd3-4c96-ab69-7605b269b327 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:06.112355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee0560-95a4-419e-a339-4f4511d2dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:06.113025 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0560-95a4-419e-a339-4f4511d2dc8c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.113328 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0560-95a4-419e-a339-4f4511d2dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.113601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0560-95a4-419e-a339-4f4511d2dc8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.114381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee0560-95a4-419e-a339-4f4511d2dc8c None admin] RBAC: Adding query filter params (interface=public, service_id=082b7cf85b424b56adb1364e609a3a17, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:06.114690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee0560-95a4-419e-a339-4f4511d2dc8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:06.116051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee0560-95a4-419e-a339-4f4511d2dc8c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:06.121435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 511/1040] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:01:05 2022] GET /identity/v3/endpoints?service_id=082b7cf85b424b56adb1364e609a3a17&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:06.126845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:06.186039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1865f6-873f-40dd-bd57-8cc26c6994ab None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.186428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1865f6-873f-40dd-bd57-8cc26c6994ab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.186699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1865f6-873f-40dd-bd57-8cc26c6994ab None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.188236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 512/1041] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:01:06.193034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d6d0-7327-492f-9e26-7483e0ce7d30 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.193360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d6d0-7327-492f-9e26-7483e0ce7d30 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.193634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67d6d0-7327-492f-9e26-7483e0ce7d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.211760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-da67d6d0-7327-492f-9e26-7483e0ce7d30 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:06.213194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-da67d6d0-7327-492f-9e26-7483e0ce7d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:06.229770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67daf480-d5de-4349-a7eb-ad9163b5ecdb None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:06.266339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:06.267305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.267772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.268051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.268751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:06.270174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:06.277633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 530/1042] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:01:06 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:01:06.289117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 513/1043] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:06.307792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:06.377574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:06.408446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0210e0-0d70-42b4-ab35-5864df731248 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.408995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0210e0-0d70-42b4-ab35-5864df731248 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.408995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0210e0-0d70-42b4-ab35-5864df731248 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.410697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 514/1044] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:06.416523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e7aed-3919-4db7-810d-2af85e2098a9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.417151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e7aed-3919-4db7-810d-2af85e2098a9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.417624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e7aed-3919-4db7-810d-2af85e2098a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.423914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:06.426273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be11485-02d8-4d77-a8f7-bc7ae41b9845 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:06.440412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f90e7aed-3919-4db7-810d-2af85e2098a9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:06.442531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f90e7aed-3919-4db7-810d-2af85e2098a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:06.468606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:06.469854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.470363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.470680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.471868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:06.473600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:06.477203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 531/1045] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:06.480811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:06.521190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 515/1046] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:06.540098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e7aed-3919-4db7-810d-2af85e2098a9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:06.590251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d6322-70c2-419c-9117-48e5fd22c78d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:06.621587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:06.629239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:06.629919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.630205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.630461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.631222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:06.632659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:06.643916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e7aed-3919-4db7-810d-2af85e2098a9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:06.655244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 532/1047] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:01:06 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:06.681987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:06.682838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.683124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.683396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.685923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:06.697195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:06.706061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:06.707135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:06.707996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 516/1048] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 07:01:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:06.713876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:06.827914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e8106-7a68-452d-99e4-d1adfab8bd79 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:06.865361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:06.866045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:06.866314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:06.866582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:06.867331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:06.867628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:06.869110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:06.877786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 533/1049] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 07:01:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:06.883681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:06.998077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68452-7c33-4b2e-8176-d8e88c576d51 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:07.033681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:07.034480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:07.034755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:07.035023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:07.046086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:07.047527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:07.064619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:07.065589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:07.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:01:07.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:07.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:07.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:07.066497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 517/1050] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:07.071669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:07.123284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:07.178041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827be9-5792-4ea5-b114-6e4ae0b6272d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:07.213475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:07.214141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:07.214418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:07.214681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:07.215389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:07.215686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:07.217096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:07.229375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 534/1051] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:07.235029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:07.281606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:07.335641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb8bb2-f8d8-40a3-b744-4d6de9770610 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:07.371325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:07.372134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:07.372415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:07.372687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:07.380935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:07.382443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:01:07.398248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:07.399327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:07.400286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 518/1052] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:01:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:07.409096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:07.509363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50053c0d-1fc0-496a-859c-c1a8d717efe8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:07.520342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:07.545486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:07.546176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:07.546456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:07.546724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:07.547550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:07.547854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:07.549301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:07.571131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 535/1053] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:01:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:07.572737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:07.675214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78956f-8cec-4b15-8ef0-f40396bc6958 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:07.682255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:07.712048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:07.712845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:07.713151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:07.713420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:07.723801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:07.725239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:07.743356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:07.744330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:07.745700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:01:07.745700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:07.745700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:07.745700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:07.749668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 519/1054] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:07.750776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:07.856048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc876427-8498-4292-94d7-3d7b526cac17 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:07.890971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:07.891636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:07.891913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:07.892187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:07.892910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:07.893224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:07.894607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:07.910707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 536/1055] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:07.911923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.019295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7558fd6-845a-496a-8969-18dd990ac5e2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:08.053758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:08.054556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.054839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.055111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.062151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:08.063616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:01:08.077823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:08.078742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:01:08.079653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:08.080830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 520/1056] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:08.084912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.115817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:08.191117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1069-ffb0-40aa-b517-0f13b125668f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:08.226965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200635e4-339b-44e9-b65e-3e111f61081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:08.227657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200635e4-339b-44e9-b65e-3e111f61081d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.227939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200635e4-339b-44e9-b65e-3e111f61081d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.228213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200635e4-339b-44e9-b65e-3e111f61081d None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.228994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200635e4-339b-44e9-b65e-3e111f61081d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:08.229330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200635e4-339b-44e9-b65e-3e111f61081d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:08.230762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200635e4-339b-44e9-b65e-3e111f61081d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:08.243795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 537/1057] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:08.249602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.269829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:08.299244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad394-f8c4-490d-9db1-d6057e325fed None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.299507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad394-f8c4-490d-9db1-d6057e325fed None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.299784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad394-f8c4-490d-9db1-d6057e325fed None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.301454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 538/1058] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:08.306485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be553e-bebf-47cd-a910-a1e680d0c406 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.306774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be553e-bebf-47cd-a910-a1e680d0c406 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.307054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be553e-bebf-47cd-a910-a1e680d0c406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.324089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-19be553e-bebf-47cd-a910-a1e680d0c406 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:08.325305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-19be553e-bebf-47cd-a910-a1e680d0c406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:08.359222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdc5f5-327c-4879-9405-cf2b75879ee2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:08.396108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:08.397110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 539/1059] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:08.397241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.397241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.397519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/95c860dcbf3b40bbb7427a417357c68a/roles/0e26787cf1fa43b5950d51f66216162e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.399807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=95c860dcbf3b40bbb7427a417357c68a, role_id=0e26787cf1fa43b5950d51f66216162e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:08.401282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:08.418305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 521/1060] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:01:08 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/95c860dcbf3b40bbb7427a417357c68a/roles/0e26787cf1fa43b5950d51f66216162e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:01:08.426533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be553e-bebf-47cd-a910-a1e680d0c406 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.448975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:08.540579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be553e-bebf-47cd-a910-a1e680d0c406 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:08.581690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adec4ea-b321-4c32-9268-2b050f145c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:08.582496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adec4ea-b321-4c32-9268-2b050f145c3e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.582773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adec4ea-b321-4c32-9268-2b050f145c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.583045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adec4ea-b321-4c32-9268-2b050f145c3e None admin] PATH_INFO: `/v3/services/image` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.583698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adec4ea-b321-4c32-9268-2b050f145c3e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:08.585076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adec4ea-b321-4c32-9268-2b050f145c3e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-2adec4ea-b321-4c32-9268-2b050f145c3e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:08.597155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:08.598424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:08.599570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 540/1061] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Jul 12 07:01:08 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:08.605975 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.695357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:08.722376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dc51a-808a-4da8-bef8-efeeb2893dff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:08.744753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce243b09-0326-4ae7-9a2e-c1b3a960390d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.745119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce243b09-0326-4ae7-9a2e-c1b3a960390d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.745340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce243b09-0326-4ae7-9a2e-c1b3a960390d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.746858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 541/1062] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:01:08.751597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b604a2-4cb3-4441-be17-d64d0b0bdd46 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.751877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b604a2-4cb3-4441-be17-d64d0b0bdd46 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.752141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b604a2-4cb3-4441-be17-d64d0b0bdd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.760453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:08.761165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.761448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.761720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.762439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:08.762745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:08.764545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:08.769975 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 522/1063] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:08.770551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-11b604a2-4cb3-4441-be17-d64d0b0bdd46 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:08.771760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-11b604a2-4cb3-4441-be17-d64d0b0bdd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:08.775361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.848001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 542/1064] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:08.871108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b604a2-4cb3-4441-be17-d64d0b0bdd46 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.883975 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2461bd-8c82-4ec1-a329-b23154955fa3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:08.888576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:08.920248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:08.921003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:08.921537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:08.921892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:08.922578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] RBAC: Adding query filter params (type=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:08.922887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:08.924268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:08.933865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 523/1065] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:08.935097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:08.944776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:08.988163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b604a2-4cb3-4441-be17-d64d0b0bdd46 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:09.028130 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:09.028936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:09.029251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:09.029528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:09.040025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:09.045134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcb3a1-578f-4d87-8740-467122fcca69 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:09.047751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:09.048737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:09.049654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:09.053869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 543/1066] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:09.054747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:09.090304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:09.091138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:09.091545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:09.091886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:09.092724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:09.094601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:09.117720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 524/1067] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:01:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:01:09.147760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:09.185162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a85ebb-33f0-442e-8cad-a22bcb70e2dd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:09.230120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:09.230824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:09.231090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:09.231363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:09.232115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:09.232419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:09.233944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:09.248705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 544/1068] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:09.250159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:09.316510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:09.358444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aff121-b33a-4545-a262-7d2cded604c9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:09.394017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:09.394887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:09.395171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:09.395437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] PATH_INFO: `/v3/users/46f5a09fec6349b1b6720c42fdba65aa` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:09.409616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] RBAC: Authorizing `identity:get_user(user_id=46f5a09fec6349b1b6720c42fdba65aa)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:09.411056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:09.412914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 525/1069] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:09 2022] GET /identity/v3/users/46f5a09fec6349b1b6720c42fdba65aa => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:09.418512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:09.520082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3e136-2da2-4e32-843e-ec5caba434f0 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:09.550702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:09.558542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:09.559197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:09.559475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:09.559748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:09.630100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:09.631549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:09.729770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 545/1070] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:09.735582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:09.834540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516abaf6-81a9-4773-ba07-ea0ee6706e1b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:09.854751 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:09.871409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b1b02-f503-465f-9b34-84fe34d42616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:09.872283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b1b02-f503-465f-9b34-84fe34d42616 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:09.872561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b1b02-f503-465f-9b34-84fe34d42616 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:09.872829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b1b02-f503-465f-9b34-84fe34d42616 None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:09.874122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b1b02-f503-465f-9b34-84fe34d42616 None admin] RBAC: Authorizing `identity:get_project(project_id=995a8687b40b40e49f3211f1d6730e1b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:09.875614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b1b02-f503-465f-9b34-84fe34d42616 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:09.877701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 526/1071] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:09 2022] GET /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:09.883602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:09.913643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:09.986873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8af5a-c61e-4c2b-8bcb-0834e8ace3da None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:10.026195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13e04d-2426-4ca3-a78d-f4e57d868d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:10.026844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13e04d-2426-4ca3-a78d-f4e57d868d8b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:10.027120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13e04d-2426-4ca3-a78d-f4e57d868d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:10.027388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13e04d-2426-4ca3-a78d-f4e57d868d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:10.028216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13e04d-2426-4ca3-a78d-f4e57d868d8b None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=995a8687b40b40e49f3211f1d6730e1b, user.id=46f5a09fec6349b1b6720c42fdba65aa) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:10.028522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13e04d-2426-4ca3-a78d-f4e57d868d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:10.029968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13e04d-2426-4ca3-a78d-f4e57d868d8b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:10.035740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 546/1072] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:09 2022] GET /identity/v3/role_assignments?user.id=46f5a09fec6349b1b6720c42fdba65aa&scope.project.id=995a8687b40b40e49f3211f1d6730e1b&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:10.436442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a04ea-75c8-42f8-af96-2806821e1a42 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:10.436814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a04ea-75c8-42f8-af96-2806821e1a42 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:10.437121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a04ea-75c8-42f8-af96-2806821e1a42 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:10.438553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 527/1073] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:10.443801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c64fe-49a2-4558-83eb-4c27f033262f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:10.444091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c64fe-49a2-4558-83eb-4c27f033262f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:10.444365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c64fe-49a2-4558-83eb-4c27f033262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:10.462523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bd3c64fe-49a2-4558-83eb-4c27f033262f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:10.464030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bd3c64fe-49a2-4558-83eb-4c27f033262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:10.535920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 547/1074] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:10.558700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a04ea-75c8-42f8-af96-2806821e1a42 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:10.662542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a04ea-75c8-42f8-af96-2806821e1a42 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:10.697633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:10.698723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:10.698997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:10.699271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:10.709939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:10.718558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:10.719570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 07:01:10.720472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:10.725434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 528/1075] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 07:01:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:10.731322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c64fe-49a2-4558-83eb-4c27f033262f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:10.758461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:10.837267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c64fe-49a2-4558-83eb-4c27f033262f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:10.870801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:10.871469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:10.871748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:10.872015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:10.872746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:10.873060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:10.874566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:10.883129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 548/1076] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 07:01:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:10.888688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:10.938758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:10.995222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53e9a6-c365-4ab8-8bbd-e120ee0d07fb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:11.033168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989373a7-dcc6-4e65-8f65-664e41cdb7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:11.033966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989373a7-dcc6-4e65-8f65-664e41cdb7fc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.034333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989373a7-dcc6-4e65-8f65-664e41cdb7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.034522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989373a7-dcc6-4e65-8f65-664e41cdb7fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.045416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989373a7-dcc6-4e65-8f65-664e41cdb7fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:11.046980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989373a7-dcc6-4e65-8f65-664e41cdb7fc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-989373a7-dcc6-4e65-8f65-664e41cdb7fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:11.064469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:11.065497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:11.066416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:01:11.066416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:11.066416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:11.066416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:11.069428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 529/1077] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:11.071747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.169255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:11.186607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac900efb-9d5d-4582-9ff5-907c4003781d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:11.196676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28959d1b-ccf0-48ea-b6a9-c4ce970ac2fa None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.197341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28959d1b-ccf0-48ea-b6a9-c4ce970ac2fa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.197621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28959d1b-ccf0-48ea-b6a9-c4ce970ac2fa None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.199791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 530/1078] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:11.207100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df709d25-6c3a-48a0-a868-85a522adc7fa None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.207491 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df709d25-6c3a-48a0-a868-85a522adc7fa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.207790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df709d25-6c3a-48a0-a868-85a522adc7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.226535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-df709d25-6c3a-48a0-a868-85a522adc7fa None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:11.226764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:11.227205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.227488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.227734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-df709d25-6c3a-48a0-a868-85a522adc7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:11.227836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.228479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:11.228789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:11.230206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:11.242548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 549/1079] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:11.247967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.301088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 531/1080] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:11.322905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df709d25-6c3a-48a0-a868-85a522adc7fa None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.360898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:11.362142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082f2c-dcaa-4ca8-8ba8-56365852a8da None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:11.398935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:11.399738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.400023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.400300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.408356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:11.409843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:01:11.424238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:11.425255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:11.426204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 550/1081] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:11.436179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.443558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:11.445021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df709d25-6c3a-48a0-a868-85a522adc7fa None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:11.492222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:11.492944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.493204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.493488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.545526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561d857-cdf2-481a-844b-f0ec62da77ba None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:11.568268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:11.569842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:11.587262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:11.587947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.588219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.588499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.589374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:11.589684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:11.591121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:11.602598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:11.610381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 551/1082] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:11.615988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.677716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:11.688306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 532/1083] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:11.689991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.719047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d046c-8265-4ced-8653-6db2a7873c04 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:11.757173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:11.758533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.758819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.759090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.794991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cee76-b3c5-4efd-b0e6-84ac50f94e72 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:11.824183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:11.830214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:11.831687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:11.831890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:11.832657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:11.832944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:11.833249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:11.846091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:11.847742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:11.865707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:11.866627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:11.867539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:01:11.867539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:11.867539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:11.867539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:11.867539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 533/1084] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:11.873149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.901357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:11.943306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 552/1085] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 07:01:11.949612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:11.984004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d58bce-c568-4c50-ac3a-a494a3c484e3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.024507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d56e9-17fd-47eb-8040-f2069f6d070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.025547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d56e9-17fd-47eb-8040-f2069f6d070f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.025829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d56e9-17fd-47eb-8040-f2069f6d070f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.026103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d56e9-17fd-47eb-8040-f2069f6d070f None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.026850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d56e9-17fd-47eb-8040-f2069f6d070f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:12.027158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d56e9-17fd-47eb-8040-f2069f6d070f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.028543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d56e9-17fd-47eb-8040-f2069f6d070f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.049079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 534/1086] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:12.061085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d78356-7f23-41ee-ac8c-3cabbc640dd7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.061401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d78356-7f23-41ee-ac8c-3cabbc640dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.061688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d78356-7f23-41ee-ac8c-3cabbc640dd7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.063321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 535/1087] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:01:12.068958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24812cbe-2aea-466c-a96b-65c1190f959a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.069194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140f6bc-7769-45c8-b3a2-3e8dae7e6901 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.069417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140f6bc-7769-45c8-b3a2-3e8dae7e6901 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.069609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140f6bc-7769-45c8-b3a2-3e8dae7e6901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.088989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5140f6bc-7769-45c8-b3a2-3e8dae7e6901 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:12.090265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5140f6bc-7769-45c8-b3a2-3e8dae7e6901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:12.113697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.114521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.114802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.115073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.126451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.127954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.131934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:12.151663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:12.152689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:12.154558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:01:12.154558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:12.154558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:01:12.154558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:12.154558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 553/1088] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:12.159244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.173506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 536/1089] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:12.179317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:12.196852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140f6bc-7769-45c8-b3a2-3e8dae7e6901 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.275391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83326-7a68-4ac9-a15f-4703289ab04b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.310728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140f6bc-7769-45c8-b3a2-3e8dae7e6901 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.317787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.318466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.318753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.319030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.319762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:12.320070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.322102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.336801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 554/1090] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:12.342353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.347260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.348065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.348356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.348629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.359078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:12.366569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:12.367610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:12.368622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 537/1091] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:12.373829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.403733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:12.418237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:12.444320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f185d-70fc-4fcd-83b1-4493aa681590 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.473549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5f9ab-6644-4048-b319-c0b3f5b38a00 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.478350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.479150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.479423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.479686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.486287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.487758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:01:12.502340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:12.503523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:12.504604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 555/1092] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:01:12.509513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.509652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.510237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.510519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.510788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.511541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:12.511844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.513583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.524203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 538/1093] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:12.530220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.611375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06379de2-10ed-49fc-8dd4-f1c85e3b4d0a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.626766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:12.633814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc181dc-f5c6-43fa-9598-7ef989c5436d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.645569 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.646229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.646500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.646786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.647594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:12.647897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.649356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.651819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:12.661611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 556/1094] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:12.667555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.667730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.668413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.668689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.668958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] PATH_INFO: `/v3/users/46f5a09fec6349b1b6720c42fdba65aa` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.670179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] RBAC: Authorizing `identity:get_user(user_id=46f5a09fec6349b1b6720c42fdba65aa)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.671726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.678311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 539/1095] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/users/46f5a09fec6349b1b6720c42fdba65aa => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:12.679253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.770163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92524-1de8-49bc-b2ae-08c43140bf1a None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.783824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a2750-64d4-4b8d-b2ca-2b1e3c16b12f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.818807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.819669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.819949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.820231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.821455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] RBAC: Authorizing `identity:get_project(project_id=995a8687b40b40e49f3211f1d6730e1b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.821738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91964f04-b657-465c-b93f-f3b64566a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.822395 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91964f04-b657-465c-b93f-f3b64566a795 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.822673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91964f04-b657-465c-b93f-f3b64566a795 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.822910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.823007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91964f04-b657-465c-b93f-f3b64566a795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.824246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91964f04-b657-465c-b93f-f3b64566a795 None admin] RBAC: Adding query filter params (role.id=0e26787cf1fa43b5950d51f66216162e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=95c860dcbf3b40bbb7427a417357c68a) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:12.824559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91964f04-b657-465c-b93f-f3b64566a795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.828952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 540/1096] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:12.829131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91964f04-b657-465c-b93f-f3b64566a795 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:12.831284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:12.833940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 557/1097] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:12 2022] GET /identity/v3/role_assignments?user.id=95c860dcbf3b40bbb7427a417357c68a&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=0e26787cf1fa43b5950d51f66216162e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:12.867499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:12.946409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ccb0b-9064-4572-bbaf-a8e123fed554 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:12.982275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d3f15-1f29-4651-be1a-d5a4512371c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:12.983177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d3f15-1f29-4651-be1a-d5a4512371c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:12.983609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d3f15-1f29-4651-be1a-d5a4512371c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:12.984029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d3f15-1f29-4651-be1a-d5a4512371c0 None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b/users/46f5a09fec6349b1b6720c42fdba65aa/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:12.986164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d3f15-1f29-4651-be1a-d5a4512371c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=995a8687b40b40e49f3211f1d6730e1b, user_id=46f5a09fec6349b1b6720c42fdba65aa, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:12.988060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d3f15-1f29-4651-be1a-d5a4512371c0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:13.009079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 541/1098] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:01:12 2022] PUT /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b/users/46f5a09fec6349b1b6720c42fdba65aa/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:01:15.054270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab79f2-a0c0-40aa-b07d-ba7257ddccbb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:15.054858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab79f2-a0c0-40aa-b07d-ba7257ddccbb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:15.054858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab79f2-a0c0-40aa-b07d-ba7257ddccbb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:15.057090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 558/1099] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:15.065082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365b472-8c1c-41f0-9838-b653d5acf727 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:15.065537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365b472-8c1c-41f0-9838-b653d5acf727 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:15.065626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365b472-8c1c-41f0-9838-b653d5acf727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:15.084366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9365b472-8c1c-41f0-9838-b653d5acf727 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:15.085618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9365b472-8c1c-41f0-9838-b653d5acf727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:15.177816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 542/1100] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:15.212659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab79f2-a0c0-40aa-b07d-ba7257ddccbb None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:15.295259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:15.348136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab79f2-a0c0-40aa-b07d-ba7257ddccbb None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:15.392266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:15.393086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:15.393376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:15.393679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:15.404451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:15.412840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:15.413936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:15.415111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 559/1101] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:15.423551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365b472-8c1c-41f0-9838-b653d5acf727 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:15.451074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:15.549832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365b472-8c1c-41f0-9838-b653d5acf727 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:15.590115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:15.591699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:15.591988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:15.592263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:15.593112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:15.593385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:15.594759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:15.604799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 543/1102] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:15.614433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:15.728026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81121e-c8a1-46f0-b13d-30a74456324b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:15.750849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:15.765632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:15.766505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:15.766782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:15.767052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] PATH_INFO: `/v3/users/46f5a09fec6349b1b6720c42fdba65aa` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:15.768236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=46f5a09fec6349b1b6720c42fdba65aa)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:15.769713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:15.771351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 560/1103] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:15 2022] GET /identity/v3/users/46f5a09fec6349b1b6720c42fdba65aa => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:15.780221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:15.868505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:15.890772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4dac2-2a61-440e-a02c-c4115a68ea65 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:15.929723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:15.930374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:15.931035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:15.931308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:16.006397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:16.007880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:16.098387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:16.119356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 544/1104] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:16.126224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:16.252639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6cf37-9061-4481-b0fb-de59bbc2d1c6 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:16.289627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650ac4-889e-485e-8e01-53ddf4f2c828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:16.290490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650ac4-889e-485e-8e01-53ddf4f2c828 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:16.290767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650ac4-889e-485e-8e01-53ddf4f2c828 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:16.291011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650ac4-889e-485e-8e01-53ddf4f2c828 None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:16.292325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650ac4-889e-485e-8e01-53ddf4f2c828 None admin] RBAC: Authorizing `identity:get_project(project_id=995a8687b40b40e49f3211f1d6730e1b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:16.293796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650ac4-889e-485e-8e01-53ddf4f2c828 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:16.295526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 561/1105] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:16 2022] GET /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:16.302216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:16.404231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc9348-befa-43eb-ae92-f8430529c211 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:16.443690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efbf9e-b2a7-4689-a6ba-8b59964226ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:16.444345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efbf9e-b2a7-4689-a6ba-8b59964226ff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:16.444629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efbf9e-b2a7-4689-a6ba-8b59964226ff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:16.444901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efbf9e-b2a7-4689-a6ba-8b59964226ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:16.445774 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efbf9e-b2a7-4689-a6ba-8b59964226ff None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=995a8687b40b40e49f3211f1d6730e1b, user.id=46f5a09fec6349b1b6720c42fdba65aa) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:16.446092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efbf9e-b2a7-4689-a6ba-8b59964226ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:16.447536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efbf9e-b2a7-4689-a6ba-8b59964226ff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:16.455219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 545/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:16 2022] GET /identity/v3/role_assignments?user.id=46f5a09fec6349b1b6720c42fdba65aa&scope.project.id=995a8687b40b40e49f3211f1d6730e1b&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:18.496260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931029-9c07-4d19-8f68-ce8df9080fd1 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:18.496819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931029-9c07-4d19-8f68-ce8df9080fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:18.496819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac931029-9c07-4d19-8f68-ce8df9080fd1 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:18.498221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 562/1107] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:18.504564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff879-c7a6-4213-959d-951fcaa834c9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:18.505353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff879-c7a6-4213-959d-951fcaa834c9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:18.505476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ff879-c7a6-4213-959d-951fcaa834c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:18.527274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f83ff879-c7a6-4213-959d-951fcaa834c9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:18.528459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f83ff879-c7a6-4213-959d-951fcaa834c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:18.574330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:18.613008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 546/1108] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:18.637432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac931029-9c07-4d19-8f68-ce8df9080fd1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:18.714474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:18.761860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac931029-9c07-4d19-8f68-ce8df9080fd1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:18.801322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec67ee2-4890-48a8-91f4-5af53c553b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:18.802108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67ee2-4890-48a8-91f4-5af53c553b42 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:18.802380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67ee2-4890-48a8-91f4-5af53c553b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:18.802650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67ee2-4890-48a8-91f4-5af53c553b42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:18.803930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec67ee2-4890-48a8-91f4-5af53c553b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:18.805381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec67ee2-4890-48a8-91f4-5af53c553b42 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:18.807030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 563/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:18.814256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ff879-c7a6-4213-959d-951fcaa834c9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:18.921594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ff879-c7a6-4213-959d-951fcaa834c9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:18.957794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bda30-232b-428a-938f-8028c57ba88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:18.958446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bda30-232b-428a-938f-8028c57ba88f None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:18.958723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bda30-232b-428a-938f-8028c57ba88f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:18.958998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bda30-232b-428a-938f-8028c57ba88f None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:18.959710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bda30-232b-428a-938f-8028c57ba88f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:18.961163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bda30-232b-428a-938f-8028c57ba88f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:18.983281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 547/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:01:18 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:01:21.054756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc698-908a-4e9c-b6d1-6b3c6b6959c7 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:21.055278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc698-908a-4e9c-b6d1-6b3c6b6959c7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:21.055278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc698-908a-4e9c-b6d1-6b3c6b6959c7 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:21.056722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 564/1111] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:21.063432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9cde-e030-493c-a63c-7f64cb08849e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:21.064127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9cde-e030-493c-a63c-7f64cb08849e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:21.064351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9cde-e030-493c-a63c-7f64cb08849e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:21.087390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cf9f9cde-e030-493c-a63c-7f64cb08849e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:21.088582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cf9f9cde-e030-493c-a63c-7f64cb08849e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:21.116734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:21.163757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 548/1112] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:21.187308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc698-908a-4e9c-b6d1-6b3c6b6959c7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:21.303214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc698-908a-4e9c-b6d1-6b3c6b6959c7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:21.327225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:21.340062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:21.340920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:21.341229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:21.341500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] PATH_INFO: `/v3/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:21.343106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] RBAC: Authorizing `identity:get_role(role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:21.344548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:21.346069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 565/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:21 2022] GET /identity/v3/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:21.352187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f9cde-e030-493c-a63c-7f64cb08849e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:21.456489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f9cde-e030-493c-a63c-7f64cb08849e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:21.491297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:21.492163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:21.492440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:21.492707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] PATH_INFO: `/v3/users/74e8287a75b943348a2df5e8a1fa8fa9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:21.506296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] RBAC: Authorizing `identity:get_user(user_id=74e8287a75b943348a2df5e8a1fa8fa9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:21.507729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:21.509387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 549/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:21 2022] GET /identity/v3/users/74e8287a75b943348a2df5e8a1fa8fa9 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:21.518073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:21.625670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afc508-8025-482a-939e-7ca62b766300 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:21.661498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:21.662142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:21.662426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:21.662700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:21.725150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:21.737772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:21.739359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:21.848571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 566/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:21.857618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:21.880317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:21.961886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47aacc-ee60-47c5-9cf4-6ab62d29c1c9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:21.999664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb6ba1-4e82-4012-984e-a05621b869a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:22.000610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb6ba1-4e82-4012-984e-a05621b869a1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:22.000858 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb6ba1-4e82-4012-984e-a05621b869a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:22.001295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb6ba1-4e82-4012-984e-a05621b869a1 None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:22.002547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb6ba1-4e82-4012-984e-a05621b869a1 None admin] RBAC: Authorizing `identity:get_project(project_id=995a8687b40b40e49f3211f1d6730e1b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:22.003990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb6ba1-4e82-4012-984e-a05621b869a1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:22.005747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 550/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:21 2022] GET /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:22.012080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:22.118162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:22.128107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99f755-5faf-4e63-a93b-07904f71470c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:22.163478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2542c-55e8-4e0a-b694-f7e925145322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:22.164059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2542c-55e8-4e0a-b694-f7e925145322 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:22.164332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2542c-55e8-4e0a-b694-f7e925145322 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:22.164602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2542c-55e8-4e0a-b694-f7e925145322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:22.165460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2542c-55e8-4e0a-b694-f7e925145322 None admin] RBAC: Adding query filter params (role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=995a8687b40b40e49f3211f1d6730e1b, user.id=74e8287a75b943348a2df5e8a1fa8fa9) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:22.165772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2542c-55e8-4e0a-b694-f7e925145322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:22.167190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2542c-55e8-4e0a-b694-f7e925145322 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:22.172898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 567/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:22 2022] GET /identity/v3/role_assignments?user.id=74e8287a75b943348a2df5e8a1fa8fa9&scope.project.id=995a8687b40b40e49f3211f1d6730e1b&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:24.151833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143938d-c872-492b-aaea-8901a4b1e380 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:24.152435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143938d-c872-492b-aaea-8901a4b1e380 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:24.152435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143938d-c872-492b-aaea-8901a4b1e380 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:24.154549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 551/1118] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:24.160980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3bb60-30b8-4b87-bc9c-139464add088 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:24.161488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3bb60-30b8-4b87-bc9c-139464add088 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:24.161587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3bb60-30b8-4b87-bc9c-139464add088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:24.178908 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5de3bb60-30b8-4b87-bc9c-139464add088 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:24.180080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5de3bb60-30b8-4b87-bc9c-139464add088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:24.248211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 568/1119] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:24.270869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143938d-c872-492b-aaea-8901a4b1e380 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:24.367348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:24.386021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143938d-c872-492b-aaea-8901a4b1e380 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:24.424555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:24.425437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:24.425719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:24.425994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] PATH_INFO: `/v3/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:24.427490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] RBAC: Authorizing `identity:get_role(role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:24.428940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:24.430364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 552/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:24 2022] GET /identity/v3/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:24.436545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3bb60-30b8-4b87-bc9c-139464add088 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:24.537523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:24.553776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3bb60-30b8-4b87-bc9c-139464add088 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:24.591009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:24.591885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:24.592163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:24.592433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] PATH_INFO: `/v3/users/74e8287a75b943348a2df5e8a1fa8fa9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:24.593867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] RBAC: Authorizing `identity:get_user(user_id=74e8287a75b943348a2df5e8a1fa8fa9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:24.595220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:24.596903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 569/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:24 2022] GET /identity/v3/users/74e8287a75b943348a2df5e8a1fa8fa9 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:24.603202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:24.709001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9ab89-4468-498c-9e79-906f05264536 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:24.744532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586558-56ed-41a9-b5f0-6505d804795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:24.745402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586558-56ed-41a9-b5f0-6505d804795a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:24.745678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586558-56ed-41a9-b5f0-6505d804795a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:24.745943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586558-56ed-41a9-b5f0-6505d804795a None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:24.747227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d586558-56ed-41a9-b5f0-6505d804795a None admin] RBAC: Authorizing `identity:get_project(project_id=995a8687b40b40e49f3211f1d6730e1b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:24.748653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d586558-56ed-41a9-b5f0-6505d804795a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:24.750511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 553/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:24 2022] GET /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:24.756949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:24.859496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e589bb-526f-4e6f-a1be-aed60c1592dc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:24.895621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca019e-940f-42a8-99bc-5ee456465a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:24.896506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca019e-940f-42a8-99bc-5ee456465a85 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:24.896784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca019e-940f-42a8-99bc-5ee456465a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:24.897084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca019e-940f-42a8-99bc-5ee456465a85 None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b/users/74e8287a75b943348a2df5e8a1fa8fa9/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:24.899394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca019e-940f-42a8-99bc-5ee456465a85 None admin] RBAC: Authorizing `identity:create_grant(project_id=995a8687b40b40e49f3211f1d6730e1b, user_id=74e8287a75b943348a2df5e8a1fa8fa9, role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:24.900831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca019e-940f-42a8-99bc-5ee456465a85 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:24.916676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 570/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:01:24 2022] PUT /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b/users/74e8287a75b943348a2df5e8a1fa8fa9/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:01:26.886146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7072f-9e4c-41d5-b3a6-d13793a3e811 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:26.886741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7072f-9e4c-41d5-b3a6-d13793a3e811 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:26.886741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7072f-9e4c-41d5-b3a6-d13793a3e811 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:26.888090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 554/1124] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:26.894435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe07fad-e2d5-41d6-a020-0cf976175d13 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:26.895130 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe07fad-e2d5-41d6-a020-0cf976175d13 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:26.895303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe07fad-e2d5-41d6-a020-0cf976175d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:26.923980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cfe07fad-e2d5-41d6-a020-0cf976175d13 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:26.925191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cfe07fad-e2d5-41d6-a020-0cf976175d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:26.934682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:26.998988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 571/1125] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:27.023083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7072f-9e4c-41d5-b3a6-d13793a3e811 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:27.051156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:27.142235 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7072f-9e4c-41d5-b3a6-d13793a3e811 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:27.177859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:27.178715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:27.178997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:27.179273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] PATH_INFO: `/v3/roles/a1db1412a1ff4cd99bfbbd1b27350de0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:27.180787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] RBAC: Authorizing `identity:get_role(role_id=a1db1412a1ff4cd99bfbbd1b27350de0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:27.182201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:27.183609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 555/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:27 2022] GET /identity/v3/roles/a1db1412a1ff4cd99bfbbd1b27350de0 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:27.191591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe07fad-e2d5-41d6-a020-0cf976175d13 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:27.300719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe07fad-e2d5-41d6-a020-0cf976175d13 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:27.337344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:27.338222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:27.338500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:27.338775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] PATH_INFO: `/v3/users/74e8287a75b943348a2df5e8a1fa8fa9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:27.340081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=74e8287a75b943348a2df5e8a1fa8fa9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:27.341562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:27.343188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 572/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:27 2022] GET /identity/v3/users/74e8287a75b943348a2df5e8a1fa8fa9 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:27.352452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:27.461278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:27.472448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432316cb-4026-4aca-aba5-a6cb739aa7cc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:27.508101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:27.508746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:27.509023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:27.509325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:27.576964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:27.578476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:27.669426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:27.681072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 556/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:27.690379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:27.710783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:27.803120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34534-30bf-49cb-982a-6fd35976e2f5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:27.838279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196b7a2-0d57-4fb6-b3f6-1fd2f60e7545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:27.839135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b7a2-0d57-4fb6-b3f6-1fd2f60e7545 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:27.839414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b7a2-0d57-4fb6-b3f6-1fd2f60e7545 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:27.839684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196b7a2-0d57-4fb6-b3f6-1fd2f60e7545 None admin] PATH_INFO: `/v3/projects/995a8687b40b40e49f3211f1d6730e1b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:27.840899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196b7a2-0d57-4fb6-b3f6-1fd2f60e7545 None admin] RBAC: Authorizing `identity:get_project(project_id=995a8687b40b40e49f3211f1d6730e1b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:27.842376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196b7a2-0d57-4fb6-b3f6-1fd2f60e7545 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:27.844354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 573/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:27 2022] GET /identity/v3/projects/995a8687b40b40e49f3211f1d6730e1b => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:27.851319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:27.976716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516bd67-8c47-450f-90c4-0b238de18c40 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:28.016798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71e547-f671-4230-9cc9-09c1c039e280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:28.017502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71e547-f671-4230-9cc9-09c1c039e280 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:28.017790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71e547-f671-4230-9cc9-09c1c039e280 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:28.018068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71e547-f671-4230-9cc9-09c1c039e280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:28.018902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71e547-f671-4230-9cc9-09c1c039e280 None admin] RBAC: Adding query filter params (role.id=a1db1412a1ff4cd99bfbbd1b27350de0, scope.project.id=995a8687b40b40e49f3211f1d6730e1b, user.id=74e8287a75b943348a2df5e8a1fa8fa9) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:28.019214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71e547-f671-4230-9cc9-09c1c039e280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:28.020648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71e547-f671-4230-9cc9-09c1c039e280 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:28.026679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 557/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:27 2022] GET /identity/v3/role_assignments?user.id=74e8287a75b943348a2df5e8a1fa8fa9&scope.project.id=995a8687b40b40e49f3211f1d6730e1b&role.id=a1db1412a1ff4cd99bfbbd1b27350de0 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:30.048901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39da64e-60be-4117-a952-ff8f2f497da3 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:30.049473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39da64e-60be-4117-a952-ff8f2f497da3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:30.049473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39da64e-60be-4117-a952-ff8f2f497da3 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:30.050877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 574/1131] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:30.057236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041ea1c-98ed-427c-bc9e-048a784e4d84 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:30.057995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041ea1c-98ed-427c-bc9e-048a784e4d84 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:30.058138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041ea1c-98ed-427c-bc9e-048a784e4d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:30.077912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e041ea1c-98ed-427c-bc9e-048a784e4d84 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:30.079262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e041ea1c-98ed-427c-bc9e-048a784e4d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:30.132997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:30.154123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 558/1132] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:30.178094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39da64e-60be-4117-a952-ff8f2f497da3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:30.265537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:30.289205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39da64e-60be-4117-a952-ff8f2f497da3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:30.326186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b625-f13e-4a66-8a0a-efa9bb1f7b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:30.326972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b625-f13e-4a66-8a0a-efa9bb1f7b3e None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:30.327249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b625-f13e-4a66-8a0a-efa9bb1f7b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:30.327522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b625-f13e-4a66-8a0a-efa9bb1f7b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:30.328826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b625-f13e-4a66-8a0a-efa9bb1f7b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:30.330295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b625-f13e-4a66-8a0a-efa9bb1f7b3e None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:30.331948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 575/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:30.340639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041ea1c-98ed-427c-bc9e-048a784e4d84 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:30.447877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041ea1c-98ed-427c-bc9e-048a784e4d84 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:30.484010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd570e-e702-4169-9783-78c080db7281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:30.484711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd570e-e702-4169-9783-78c080db7281 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:30.484995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd570e-e702-4169-9783-78c080db7281 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:30.485307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd570e-e702-4169-9783-78c080db7281 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:30.486006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd570e-e702-4169-9783-78c080db7281 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:30.487462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd570e-e702-4169-9783-78c080db7281 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:30.519271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 559/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:01:30 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:32.598152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a84f3-70cb-4fbf-a0d7-8b2ab882a3de None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:32.598743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a84f3-70cb-4fbf-a0d7-8b2ab882a3de None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:32.598743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a84f3-70cb-4fbf-a0d7-8b2ab882a3de None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:32.600089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 576/1135] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:32.606575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b87d-e617-4a5f-9a6b-c6021630bcec None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:32.607313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b87d-e617-4a5f-9a6b-c6021630bcec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:32.607457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b87d-e617-4a5f-9a6b-c6021630bcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:32.633089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5566b87d-e617-4a5f-9a6b-c6021630bcec None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:32.634306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5566b87d-e617-4a5f-9a6b-c6021630bcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:32.668788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:32.711812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 560/1136] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:32.745918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a84f3-70cb-4fbf-a0d7-8b2ab882a3de None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:32.849328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a84f3-70cb-4fbf-a0d7-8b2ab882a3de None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:32.884717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824858-55b7-4470-8bf5-a7a39ca9d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:32.885539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824858-55b7-4470-8bf5-a7a39ca9d041 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:32.885818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824858-55b7-4470-8bf5-a7a39ca9d041 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:32.886096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824858-55b7-4470-8bf5-a7a39ca9d041 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:32.887389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96824858-55b7-4470-8bf5-a7a39ca9d041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:32.888828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96824858-55b7-4470-8bf5-a7a39ca9d041 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:32.890606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 577/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:32.896717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566b87d-e617-4a5f-9a6b-c6021630bcec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:33.001927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566b87d-e617-4a5f-9a6b-c6021630bcec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:33.038758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa696fff-4741-4e6e-9748-4cb303bd0202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:33.039425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa696fff-4741-4e6e-9748-4cb303bd0202 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:33.039710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa696fff-4741-4e6e-9748-4cb303bd0202 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:33.039983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa696fff-4741-4e6e-9748-4cb303bd0202 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:33.040692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa696fff-4741-4e6e-9748-4cb303bd0202 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:33.042169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa696fff-4741-4e6e-9748-4cb303bd0202 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:33.066860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 561/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:01:32 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:35.110938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533c9cb-30b4-4e4a-8196-c7d31d1ad9ad None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:35.111501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533c9cb-30b4-4e4a-8196-c7d31d1ad9ad None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:35.111501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533c9cb-30b4-4e4a-8196-c7d31d1ad9ad None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:35.112872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 578/1139] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:35.119322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c1976-d635-492a-aaba-b2a65929a8da None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:35.120045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c1976-d635-492a-aaba-b2a65929a8da None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:35.120200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c1976-d635-492a-aaba-b2a65929a8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:35.146027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-692c1976-d635-492a-aaba-b2a65929a8da None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:35.147211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-692c1976-d635-492a-aaba-b2a65929a8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:35.165960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:35.219402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 562/1140] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:35.243206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533c9cb-30b4-4e4a-8196-c7d31d1ad9ad None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:35.264609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:35.350503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533c9cb-30b4-4e4a-8196-c7d31d1ad9ad None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:35.388612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:35.389445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:35.389726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:35.389997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:35.401720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:35.409639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:35.410801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:35.412190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 579/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:35.419493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c1976-d635-492a-aaba-b2a65929a8da None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:35.532657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c1976-d635-492a-aaba-b2a65929a8da None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:35.571532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:35.572188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:35.572462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:35.572713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:35.573844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:35.574146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:35.575523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:35.584406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 563/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:35.590621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:35.675228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:35.700407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957e14-c186-4cc7-a0c5-8ca9113e9659 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:35.739440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:35.740320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:35.740600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:35.740872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] PATH_INFO: `/v3/users/cef09e926a2b432ebd5e09e1485149ff` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:35.754895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] RBAC: Authorizing `identity:get_user(user_id=cef09e926a2b432ebd5e09e1485149ff)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:35.756348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:35.758048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 580/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:35 2022] GET /identity/v3/users/cef09e926a2b432ebd5e09e1485149ff => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:35.764043 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:35.785483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:35.873876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819159e1-ef7d-46e9-9afc-afbc38380e52 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:35.910076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:35.910723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:35.911010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:35.911284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:35.980612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:35.982084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:36.000415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:36.090443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 564/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:36.097867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:36.203232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccffc6-5f9f-458e-b91d-c3df88c91fea None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:36.245591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756d148-e4a1-438e-bdd7-38cb6aff1844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:36.246465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756d148-e4a1-438e-bdd7-38cb6aff1844 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:36.246754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756d148-e4a1-438e-bdd7-38cb6aff1844 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:36.247031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756d148-e4a1-438e-bdd7-38cb6aff1844 None admin] PATH_INFO: `/v3/projects/0725b190f96a48928940ddbafe7cc5c8` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:36.248205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756d148-e4a1-438e-bdd7-38cb6aff1844 None admin] RBAC: Authorizing `identity:get_project(project_id=0725b190f96a48928940ddbafe7cc5c8)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:36.249682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756d148-e4a1-438e-bdd7-38cb6aff1844 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:36.252026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 581/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:36 2022] GET /identity/v3/projects/0725b190f96a48928940ddbafe7cc5c8 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:36.262001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:36.381664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2877795-5ad6-42cd-b814-2276e68a24c7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:36.398183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:36.416633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a5394-cb10-4fa2-8024-420606e9a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:36.417325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a5394-cb10-4fa2-8024-420606e9a04c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:36.417609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a5394-cb10-4fa2-8024-420606e9a04c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:36.417886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a5394-cb10-4fa2-8024-420606e9a04c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:36.418714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a5394-cb10-4fa2-8024-420606e9a04c None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=0725b190f96a48928940ddbafe7cc5c8, user.id=cef09e926a2b432ebd5e09e1485149ff) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:36.419028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a5394-cb10-4fa2-8024-420606e9a04c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:36.420461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a5394-cb10-4fa2-8024-420606e9a04c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:36.426205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 565/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:36 2022] GET /identity/v3/role_assignments?user.id=cef09e926a2b432ebd5e09e1485149ff&scope.project.id=0725b190f96a48928940ddbafe7cc5c8&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:38.419342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfb627-d52d-4380-9f37-1db4e722458e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:38.420200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfb627-d52d-4380-9f37-1db4e722458e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:38.420200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfb627-d52d-4380-9f37-1db4e722458e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:38.422332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 582/1147] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:38.428623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ea244-b953-4bf3-a0d8-eb5c01c5ce8c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:38.429388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ea244-b953-4bf3-a0d8-eb5c01c5ce8c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:38.429541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ea244-b953-4bf3-a0d8-eb5c01c5ce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:38.451217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e01ea244-b953-4bf3-a0d8-eb5c01c5ce8c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:38.452397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e01ea244-b953-4bf3-a0d8-eb5c01c5ce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:38.524773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 566/1148] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:38.548148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfb627-d52d-4380-9f37-1db4e722458e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:38.552795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:38.652144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfb627-d52d-4380-9f37-1db4e722458e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:38.687853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:38.688662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:38.688938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:38.689235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:38.700300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:38.707897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:38.708978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:38.710172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 583/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:38.715367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ea244-b953-4bf3-a0d8-eb5c01c5ce8c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:38.824322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ea244-b953-4bf3-a0d8-eb5c01c5ce8c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:38.834542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:38.866562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:38.867228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:38.867514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:38.867791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:38.868543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:38.868851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:38.870265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:38.878600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 567/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:38.884615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:38.963597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:39.004351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ba3eb-85de-4899-88b9-3d1091d33654 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:39.040013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:39.040882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:39.041200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:39.041475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] PATH_INFO: `/v3/users/cef09e926a2b432ebd5e09e1485149ff` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:39.042680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] RBAC: Authorizing `identity:get_user(user_id=cef09e926a2b432ebd5e09e1485149ff)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:39.044131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:39.045819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 584/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:38 2022] GET /identity/v3/users/cef09e926a2b432ebd5e09e1485149ff => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:39.052040 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:39.155981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41854b-c557-40c3-a787-c0c66f3497e3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:39.191213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012d8c-1b9b-4847-b864-dbaf8abc5f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:39.192069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012d8c-1b9b-4847-b864-dbaf8abc5f35 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:39.192353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012d8c-1b9b-4847-b864-dbaf8abc5f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:39.192621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012d8c-1b9b-4847-b864-dbaf8abc5f35 None admin] PATH_INFO: `/v3/projects/0725b190f96a48928940ddbafe7cc5c8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:39.193959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012d8c-1b9b-4847-b864-dbaf8abc5f35 None admin] RBAC: Authorizing `identity:get_project(project_id=0725b190f96a48928940ddbafe7cc5c8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:39.195394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012d8c-1b9b-4847-b864-dbaf8abc5f35 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:39.197096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 568/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:39 2022] GET /identity/v3/projects/0725b190f96a48928940ddbafe7cc5c8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:39.203544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:39.310700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18007-c7d0-4f11-8a9b-c1abfd917420 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:39.340857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:39.347761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3753a39-08fc-4307-890b-43d4b392ecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:39.348659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3753a39-08fc-4307-890b-43d4b392ecc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:39.348942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3753a39-08fc-4307-890b-43d4b392ecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:39.349250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3753a39-08fc-4307-890b-43d4b392ecc4 None admin] PATH_INFO: `/v3/projects/0725b190f96a48928940ddbafe7cc5c8/users/cef09e926a2b432ebd5e09e1485149ff/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:39.351245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3753a39-08fc-4307-890b-43d4b392ecc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0725b190f96a48928940ddbafe7cc5c8, user_id=cef09e926a2b432ebd5e09e1485149ff, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:39.352676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3753a39-08fc-4307-890b-43d4b392ecc4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:39.370914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 585/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:01:39 2022] PUT /identity/v3/projects/0725b190f96a48928940ddbafe7cc5c8/users/cef09e926a2b432ebd5e09e1485149ff/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:01:41.328764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480c222-6d11-4d84-ba07-ab69404b3a63 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:41.329722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480c222-6d11-4d84-ba07-ab69404b3a63 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:41.329722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480c222-6d11-4d84-ba07-ab69404b3a63 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:41.331861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 569/1154] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:41.337391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478edf8-1fdb-490d-91fa-81171b93fd94 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:41.337831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478edf8-1fdb-490d-91fa-81171b93fd94 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:41.337925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478edf8-1fdb-490d-91fa-81171b93fd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:41.356383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2478edf8-1fdb-490d-91fa-81171b93fd94 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:41.357584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2478edf8-1fdb-490d-91fa-81171b93fd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:41.430847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 586/1155] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:41.455597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480c222-6d11-4d84-ba07-ab69404b3a63 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:41.511775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:41.577783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480c222-6d11-4d84-ba07-ab69404b3a63 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:41.626317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:41.627054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:41.627340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:41.627625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:41.639619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:41.647267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:41.648348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:41.649398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 570/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:41.657124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478edf8-1fdb-490d-91fa-81171b93fd94 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:41.761943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478edf8-1fdb-490d-91fa-81171b93fd94 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:41.783190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:41.800768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:41.801475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:41.801756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:41.802030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:41.802781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:41.803089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:41.804478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:41.812822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 587/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:41.818977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:41.921911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:41.925371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd34433-78b5-42c0-991b-cbea85298ec1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:41.963028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:41.963906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:41.964190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:41.964468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] PATH_INFO: `/v3/users/cef09e926a2b432ebd5e09e1485149ff` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:41.965716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] RBAC: Authorizing `identity:get_user(user_id=cef09e926a2b432ebd5e09e1485149ff)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:41.967153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:41.968863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 571/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:41 2022] GET /identity/v3/users/cef09e926a2b432ebd5e09e1485149ff => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:41.975251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:42.083027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c25be8-c6a9-4fcc-9e51-94a1b3a2bab8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:42.125973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:42.126625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:42.126902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:42.127173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:42.177549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:42.198332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:42.199774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:42.301773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 588/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:42.308089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:42.432607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eb19d-c287-4c64-8620-dfbdb4964c15 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:42.470038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a433fc-c56a-4ff8-8aca-4393e72051ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:42.470898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a433fc-c56a-4ff8-8aca-4393e72051ac None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:42.471179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a433fc-c56a-4ff8-8aca-4393e72051ac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:42.471451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a433fc-c56a-4ff8-8aca-4393e72051ac None admin] PATH_INFO: `/v3/projects/0725b190f96a48928940ddbafe7cc5c8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:42.472674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a433fc-c56a-4ff8-8aca-4393e72051ac None admin] RBAC: Authorizing `identity:get_project(project_id=0725b190f96a48928940ddbafe7cc5c8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:42.474132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a433fc-c56a-4ff8-8aca-4393e72051ac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:42.475821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 572/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:42 2022] GET /identity/v3/projects/0725b190f96a48928940ddbafe7cc5c8 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:42.485445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:42.592760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:42.616994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c3a3-da71-4d48-922c-59eaaac9fd55 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:42.655327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd109d-2292-4843-85bf-f2ed4e12cd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:42.655999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd109d-2292-4843-85bf-f2ed4e12cd55 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:42.656304 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd109d-2292-4843-85bf-f2ed4e12cd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:42.656532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd109d-2292-4843-85bf-f2ed4e12cd55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:42.657428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd109d-2292-4843-85bf-f2ed4e12cd55 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=0725b190f96a48928940ddbafe7cc5c8, user.id=cef09e926a2b432ebd5e09e1485149ff) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:42.657741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd109d-2292-4843-85bf-f2ed4e12cd55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:42.659183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd109d-2292-4843-85bf-f2ed4e12cd55 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:42.665831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 589/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:42 2022] GET /identity/v3/role_assignments?user.id=cef09e926a2b432ebd5e09e1485149ff&scope.project.id=0725b190f96a48928940ddbafe7cc5c8&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:44.703521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e93b32-0ee6-4d7a-84b1-9981808355b6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:44.704086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e93b32-0ee6-4d7a-84b1-9981808355b6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:44.704086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e93b32-0ee6-4d7a-84b1-9981808355b6 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:44.705571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 573/1162] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:44.712143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1594a-4e6a-4e3c-a304-95f7b8d6d4c5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:44.712558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1594a-4e6a-4e3c-a304-95f7b8d6d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:44.712666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1594a-4e6a-4e3c-a304-95f7b8d6d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:44.737090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-02c1594a-4e6a-4e3c-a304-95f7b8d6d4c5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:44.738276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-02c1594a-4e6a-4e3c-a304-95f7b8d6d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:44.823642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 590/1163] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:44.847825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e93b32-0ee6-4d7a-84b1-9981808355b6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:44.889596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:44.960775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e93b32-0ee6-4d7a-84b1-9981808355b6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:44.997838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:44.998533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:44.998812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:44.999086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:45.077830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:45.079411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:45.114315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:45.180603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 574/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:45.189549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1594a-4e6a-4e3c-a304-95f7b8d6d4c5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:45.270824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:45.298788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1594a-4e6a-4e3c-a304-95f7b8d6d4c5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:45.336703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276a02f-63a4-4c64-bd13-ad8f426de11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:45.337536 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276a02f-63a4-4c64-bd13-ad8f426de11b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:45.337814 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276a02f-63a4-4c64-bd13-ad8f426de11b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:45.338087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276a02f-63a4-4c64-bd13-ad8f426de11b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:45.345215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276a02f-63a4-4c64-bd13-ad8f426de11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:45.346682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276a02f-63a4-4c64-bd13-ad8f426de11b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d276a02f-63a4-4c64-bd13-ad8f426de11b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:45.360947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:45.362175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:01:45.363397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 591/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:01:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:45.369627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:45.475379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc893c-eba3-446b-a044-63720420fbef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:45.509637 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d67574-3f50-4479-80dd-7f41a004089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:45.510287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d67574-3f50-4479-80dd-7f41a004089c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:45.510571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d67574-3f50-4479-80dd-7f41a004089c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:45.510843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d67574-3f50-4479-80dd-7f41a004089c None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:45.511564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d67574-3f50-4479-80dd-7f41a004089c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:45.511871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d67574-3f50-4479-80dd-7f41a004089c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:45.513262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d67574-3f50-4479-80dd-7f41a004089c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:45.516129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:45.521643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 575/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 07:01:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:47.507564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f03857-c717-4eb8-9018-c3dbd273d4f6 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:47.508432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f03857-c717-4eb8-9018-c3dbd273d4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:47.508432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f03857-c717-4eb8-9018-c3dbd273d4f6 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:47.510750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 592/1167] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:47.516253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec68e-bcdb-435c-abfc-45560cdb3f7a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:47.516706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec68e-bcdb-435c-abfc-45560cdb3f7a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:47.516812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec68e-bcdb-435c-abfc-45560cdb3f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:47.535238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a31ec68e-bcdb-435c-abfc-45560cdb3f7a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:47.536416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a31ec68e-bcdb-435c-abfc-45560cdb3f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:47.605716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 576/1168] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:47.629199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f03857-c717-4eb8-9018-c3dbd273d4f6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:47.739873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f03857-c717-4eb8-9018-c3dbd273d4f6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:47.775393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655c4cd-e215-41ba-bc2e-9a92b37d62f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:47.775979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655c4cd-e215-41ba-bc2e-9a92b37d62f2 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:47.776259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655c4cd-e215-41ba-bc2e-9a92b37d62f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:47.776530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655c4cd-e215-41ba-bc2e-9a92b37d62f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:47.777298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655c4cd-e215-41ba-bc2e-9a92b37d62f2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:47.778674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655c4cd-e215-41ba-bc2e-9a92b37d62f2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:47.782390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:47.797992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 593/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Tue Jul 12 07:01:47 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:49.851153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e62fe-d687-4b7f-b009-6c6409cef983 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:49.851724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e62fe-d687-4b7f-b009-6c6409cef983 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:49.851724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e62fe-d687-4b7f-b009-6c6409cef983 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:49.853088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 577/1170] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:49.859414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778082-6443-4219-982a-01e39eb37991 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:49.860125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778082-6443-4219-982a-01e39eb37991 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:49.860215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778082-6443-4219-982a-01e39eb37991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:49.884410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7c778082-6443-4219-982a-01e39eb37991 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:49.885649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7c778082-6443-4219-982a-01e39eb37991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:49.955522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 594/1171] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:49.979539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e62fe-d687-4b7f-b009-6c6409cef983 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:50.087926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e62fe-d687-4b7f-b009-6c6409cef983 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:50.117491 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:50.123815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f811995-ffe5-4a91-b5e0-ca68d946310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:50.124692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f811995-ffe5-4a91-b5e0-ca68d946310b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:50.124968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f811995-ffe5-4a91-b5e0-ca68d946310b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:50.125266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f811995-ffe5-4a91-b5e0-ca68d946310b None admin] PATH_INFO: `/v3/domains/e7d513afb82445cb965bd1747a82c2c6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:50.142538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f811995-ffe5-4a91-b5e0-ca68d946310b None admin] RBAC: Authorizing `identity:get_domain(domain_id=e7d513afb82445cb965bd1747a82c2c6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:50.144129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f811995-ffe5-4a91-b5e0-ca68d946310b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:50.148402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 578/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Jul 12 07:01:49 2022] GET /identity/v3/domains/e7d513afb82445cb965bd1747a82c2c6 => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:50.154678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c778082-6443-4219-982a-01e39eb37991 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:50.261684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c778082-6443-4219-982a-01e39eb37991 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:50.288079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:50.298049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f39661-0607-4f57-b1a5-34b4f6303152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:50.298784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f39661-0607-4f57-b1a5-34b4f6303152 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:50.299011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f39661-0607-4f57-b1a5-34b4f6303152 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:50.299291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f39661-0607-4f57-b1a5-34b4f6303152 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:50.299996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f39661-0607-4f57-b1a5-34b4f6303152 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:50.301469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f39661-0607-4f57-b1a5-34b4f6303152 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:50.330141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 595/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:01:50 2022] POST /identity/v3/projects => generated 361 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:52.339378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6bcd-4664-4a03-867d-26262672d5b3 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:52.339914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6bcd-4664-4a03-867d-26262672d5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:52.339914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6bcd-4664-4a03-867d-26262672d5b3 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:52.341269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 579/1174] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:52.347622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93cf31-b901-410d-bd19-e6a5866946d3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:52.348309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93cf31-b901-410d-bd19-e6a5866946d3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:52.348505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93cf31-b901-410d-bd19-e6a5866946d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:52.371001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6a93cf31-b901-410d-bd19-e6a5866946d3 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:52.372181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6a93cf31-b901-410d-bd19-e6a5866946d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:52.441329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 596/1175] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:52.471540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d6bcd-4664-4a03-867d-26262672d5b3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:52.579901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d6bcd-4664-4a03-867d-26262672d5b3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:52.615011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2b2b7-5d88-46ae-b4c3-209c888dac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:52.615802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2b2b7-5d88-46ae-b4c3-209c888dac63 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:52.616075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2b2b7-5d88-46ae-b4c3-209c888dac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:52.616347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2b2b7-5d88-46ae-b4c3-209c888dac63 None admin] PATH_INFO: `/v3/domains/e7d513afb82445cb965bd1747a82c2c6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:52.617649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2b2b7-5d88-46ae-b4c3-209c888dac63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e7d513afb82445cb965bd1747a82c2c6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:52.619083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2b2b7-5d88-46ae-b4c3-209c888dac63 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:52.620691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 580/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Jul 12 07:01:52 2022] GET /identity/v3/domains/e7d513afb82445cb965bd1747a82c2c6 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:52.627291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93cf31-b901-410d-bd19-e6a5866946d3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:52.753553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93cf31-b901-410d-bd19-e6a5866946d3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:52.763179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:52.792047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96dc42-f751-4acb-bdce-508d64b85c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:52.792709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96dc42-f751-4acb-bdce-508d64b85c85 None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:52.793019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96dc42-f751-4acb-bdce-508d64b85c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:52.793318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96dc42-f751-4acb-bdce-508d64b85c85 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:52.794017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96dc42-f751-4acb-bdce-508d64b85c85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:52.795444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96dc42-f751-4acb-bdce-508d64b85c85 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:52.820823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 597/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:01:52 2022] POST /identity/v3/users => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:01:54.871167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a154a-5715-4709-96a9-867b63815f65 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:54.871805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a154a-5715-4709-96a9-867b63815f65 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:54.871805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a154a-5715-4709-96a9-867b63815f65 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:54.873126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 581/1178] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:54.879048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6eb7c-08d7-47f9-91ca-b0edf0a1a09e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:54.879497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6eb7c-08d7-47f9-91ca-b0edf0a1a09e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:54.879594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6eb7c-08d7-47f9-91ca-b0edf0a1a09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:54.898532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-78d6eb7c-08d7-47f9-91ca-b0edf0a1a09e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:54.899703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-78d6eb7c-08d7-47f9-91ca-b0edf0a1a09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:54.969251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 598/1179] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:01:54.993917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a154a-5715-4709-96a9-867b63815f65 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:55.066815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:55.114656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a154a-5715-4709-96a9-867b63815f65 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:55.151212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:55.151939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:55.152231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:55.152504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:55.166199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:55.176830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:55.178052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:55.179297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 582/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:55.184308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6eb7c-08d7-47f9-91ca-b0edf0a1a09e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:55.279906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:55.288196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6eb7c-08d7-47f9-91ca-b0edf0a1a09e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:55.323966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:55.324622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:55.324901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:55.325209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:55.325953 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:55.326257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:55.327641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:55.336165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 599/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:55.342237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:55.471156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29055b2-a445-48e2-a5ba-68c2b0275144 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:55.489779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:55.510628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:55.511502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:55.511781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:55.512055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] PATH_INFO: `/v3/users/cb984f6f5acd429590cdd331fd6eb334` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:55.526831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] RBAC: Authorizing `identity:get_user(user_id=cb984f6f5acd429590cdd331fd6eb334)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:55.528288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:55.529980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 583/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:55 2022] GET /identity/v3/users/cb984f6f5acd429590cdd331fd6eb334 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:55.536175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:55.646798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7634b-8586-4b0a-916f-dcb19ffdb062 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:55.681212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:55.681848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:55.682128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:55.682402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:55.707637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:55.760800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:55.762235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:55.874850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 600/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:01:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:01:55.885121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:55.990680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab4153-f2bf-48b1-b419-67f2ad17990d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:56.026002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492266a-ff7b-4c06-a1b5-ec9c336671fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:56.026793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492266a-ff7b-4c06-a1b5-ec9c336671fb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:56.027067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492266a-ff7b-4c06-a1b5-ec9c336671fb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:56.027342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492266a-ff7b-4c06-a1b5-ec9c336671fb None admin] PATH_INFO: `/v3/projects/bbd37799d7774d16a902fabfe745b8ec` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:56.028562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492266a-ff7b-4c06-a1b5-ec9c336671fb None admin] RBAC: Authorizing `identity:get_project(project_id=bbd37799d7774d16a902fabfe745b8ec)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:56.030026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492266a-ff7b-4c06-a1b5-ec9c336671fb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:56.031757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 584/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:55 2022] GET /identity/v3/projects/bbd37799d7774d16a902fabfe745b8ec => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:56.038381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:56.098088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:56.147695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b9268-e450-4297-8828-88767c2f23c8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:56.182643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de8c4-4806-4a55-b14a-0ffe7a58e2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:56.183285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de8c4-4806-4a55-b14a-0ffe7a58e2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:56.183562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de8c4-4806-4a55-b14a-0ffe7a58e2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:56.183837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de8c4-4806-4a55-b14a-0ffe7a58e2f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:56.184661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de8c4-4806-4a55-b14a-0ffe7a58e2f6 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=bbd37799d7774d16a902fabfe745b8ec, user.id=cb984f6f5acd429590cdd331fd6eb334) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:56.184974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de8c4-4806-4a55-b14a-0ffe7a58e2f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:56.186455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de8c4-4806-4a55-b14a-0ffe7a58e2f6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:56.192008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 601/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:01:56 2022] GET /identity/v3/role_assignments?user.id=cb984f6f5acd429590cdd331fd6eb334&scope.project.id=bbd37799d7774d16a902fabfe745b8ec&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:58.182582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215cddf-07e2-4e38-a6e5-04e4af2739be None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:58.183162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215cddf-07e2-4e38-a6e5-04e4af2739be None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:58.183162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215cddf-07e2-4e38-a6e5-04e4af2739be None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:58.184498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 585/1186] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:01:58.191021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4e32b-08f9-455e-9168-ca89a0a6956d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:58.191485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4e32b-08f9-455e-9168-ca89a0a6956d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:58.191580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4e32b-08f9-455e-9168-ca89a0a6956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:58.216452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-05a4e32b-08f9-455e-9168-ca89a0a6956d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:01:58.217691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-05a4e32b-08f9-455e-9168-ca89a0a6956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:01:58.293006 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 602/1187] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:01:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:01:58.317740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215cddf-07e2-4e38-a6e5-04e4af2739be None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:58.371362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:58.429271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215cddf-07e2-4e38-a6e5-04e4af2739be None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:58.465647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:58.466437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:58.466719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:58.466983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:58.478764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:01:58.486380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:01:58.487577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:01:58.488800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 586/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:01:58.493837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4e32b-08f9-455e-9168-ca89a0a6956d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:58.549975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:58.605425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4e32b-08f9-455e-9168-ca89a0a6956d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:58.641124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:58.641790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:58.642067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:58.642383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:58.643125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:01:58.643427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:58.644811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:58.653912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 603/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:58.661344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:58.765806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765db2b0-b23b-4fea-af32-d8c3190a3ef3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:58.768189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:58.801437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:58.802230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:58.802508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:58.802782 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] PATH_INFO: `/v3/users/cb984f6f5acd429590cdd331fd6eb334` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:58.804007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] RBAC: Authorizing `identity:get_user(user_id=cb984f6f5acd429590cdd331fd6eb334)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:58.805474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:58.807110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 587/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:01:58 2022] GET /identity/v3/users/cb984f6f5acd429590cdd331fd6eb334 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:01:58.813189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:58.919815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18cfab-f828-4e15-a5cf-3acdbdd67356 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:58.932512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:01:58.957105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4a0d5-37f4-494a-aa2e-cefbe68a6728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:58.957981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4a0d5-37f4-494a-aa2e-cefbe68a6728 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:58.958258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4a0d5-37f4-494a-aa2e-cefbe68a6728 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:58.958523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4a0d5-37f4-494a-aa2e-cefbe68a6728 None admin] PATH_INFO: `/v3/projects/bbd37799d7774d16a902fabfe745b8ec` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:58.959777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4a0d5-37f4-494a-aa2e-cefbe68a6728 None admin] RBAC: Authorizing `identity:get_project(project_id=bbd37799d7774d16a902fabfe745b8ec)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:58.961310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4a0d5-37f4-494a-aa2e-cefbe68a6728 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:58.963031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 604/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:01:58 2022] GET /identity/v3/projects/bbd37799d7774d16a902fabfe745b8ec => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:01:58.969111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:01:59.076014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b844d-e476-44d7-9a53-8577ff0e2b2a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:01:59.111474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305af56-78cd-4791-82a3-d8ee04d38036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:01:59.112359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305af56-78cd-4791-82a3-d8ee04d38036 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:01:59.112638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305af56-78cd-4791-82a3-d8ee04d38036 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:01:59.112907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305af56-78cd-4791-82a3-d8ee04d38036 None admin] PATH_INFO: `/v3/projects/bbd37799d7774d16a902fabfe745b8ec/users/cb984f6f5acd429590cdd331fd6eb334/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:01:59.114969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305af56-78cd-4791-82a3-d8ee04d38036 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbd37799d7774d16a902fabfe745b8ec, user_id=cb984f6f5acd429590cdd331fd6eb334, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:01:59.116435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305af56-78cd-4791-82a3-d8ee04d38036 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:01:59.135066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 588/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:01:58 2022] PUT /identity/v3/projects/bbd37799d7774d16a902fabfe745b8ec/users/cb984f6f5acd429590cdd331fd6eb334/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:02:01.124877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f6cab-070d-4630-8c12-4ac7108efc38 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:01.125828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f6cab-070d-4630-8c12-4ac7108efc38 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:01.125828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f6cab-070d-4630-8c12-4ac7108efc38 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:01.127584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 605/1193] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:02:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:02:01.133081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548b0eb-84a2-47b5-b73e-a75a2cc9e55a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:01.133521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548b0eb-84a2-47b5-b73e-a75a2cc9e55a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:01.133629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548b0eb-84a2-47b5-b73e-a75a2cc9e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:01.151116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8548b0eb-84a2-47b5-b73e-a75a2cc9e55a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:02:01.152297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8548b0eb-84a2-47b5-b73e-a75a2cc9e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:02:01.180568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:01.227113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 589/1194] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:02:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:02:01.251787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f6cab-070d-4630-8c12-4ac7108efc38 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:01.361033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f6cab-070d-4630-8c12-4ac7108efc38 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:01.395730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:01.396520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:01.396798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:01.397096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:01.407609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:02:01.415359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:02:01.416471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:02:01.417490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 606/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:02:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:02:01.422926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548b0eb-84a2-47b5-b73e-a75a2cc9e55a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:01.522740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548b0eb-84a2-47b5-b73e-a75a2cc9e55a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:01.557597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:01.558256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:01.558532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:01.558805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:01.559543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:02:01.559850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:01.561230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:01.569399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 590/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:02:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:02:01.576007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:01.620676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:01.685107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264eebd-9b67-4d6d-a509-4f295095f4fd None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:01.720831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:01.721732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:01.722013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:01.722286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] PATH_INFO: `/v3/users/cb984f6f5acd429590cdd331fd6eb334` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:01.723489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] RBAC: Authorizing `identity:get_user(user_id=cb984f6f5acd429590cdd331fd6eb334)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:01.724950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:01.726780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 607/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 07:02:01 2022] GET /identity/v3/users/cb984f6f5acd429590cdd331fd6eb334 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:02:01.735440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:01.762067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:01.840245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ffd1e-56af-4efc-b7ae-5ae18f8fee77 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:01.875127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:01.875763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:01.876046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:01.876317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:01.945548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:01.946989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:01.979318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:02.057103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 591/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:02:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:02:02.063357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:02.169493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74647a7a-4b38-4ff5-9cc8-521f5a3f3efe None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:02.178264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:02.205568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149a81f-b106-4109-bb79-4feb5db48f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:02.206431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149a81f-b106-4109-bb79-4feb5db48f14 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:02.206709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149a81f-b106-4109-bb79-4feb5db48f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:02.206983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149a81f-b106-4109-bb79-4feb5db48f14 None admin] PATH_INFO: `/v3/projects/bbd37799d7774d16a902fabfe745b8ec` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:02.208269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149a81f-b106-4109-bb79-4feb5db48f14 None admin] RBAC: Authorizing `identity:get_project(project_id=bbd37799d7774d16a902fabfe745b8ec)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:02.209746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149a81f-b106-4109-bb79-4feb5db48f14 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:02.211456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 608/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 07:02:02 2022] GET /identity/v3/projects/bbd37799d7774d16a902fabfe745b8ec => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:02:02.217821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:02.320619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8b57e-77fb-4f15-9ffb-17a63c0042d3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:02.345038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:02.364019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451dbcc-1588-404c-a0c9-4dcc30e567e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:02.364981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451dbcc-1588-404c-a0c9-4dcc30e567e1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:02.365416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451dbcc-1588-404c-a0c9-4dcc30e567e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:02.365806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451dbcc-1588-404c-a0c9-4dcc30e567e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:02.367115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451dbcc-1588-404c-a0c9-4dcc30e567e1 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=bbd37799d7774d16a902fabfe745b8ec, user.id=cb984f6f5acd429590cdd331fd6eb334) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:02:02.367562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451dbcc-1588-404c-a0c9-4dcc30e567e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:02.369904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451dbcc-1588-404c-a0c9-4dcc30e567e1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:02.378175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 592/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:02:02 2022] GET /identity/v3/role_assignments?user.id=cb984f6f5acd429590cdd331fd6eb334&scope.project.id=bbd37799d7774d16a902fabfe745b8ec&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:02:12.200493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=12 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:12.375266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:57.833360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315d60-016a-4b7e-b53c-cbece9f9891b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:57.833977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315d60-016a-4b7e-b53c-cbece9f9891b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:57.833977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315d60-016a-4b7e-b53c-cbece9f9891b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:57.835461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 609/1201] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:02:57.840882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b382f-c5d3-4bf5-8244-1d9c326dc25f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:57.841412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b382f-c5d3-4bf5-8244-1d9c326dc25f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:57.841599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b382f-c5d3-4bf5-8244-1d9c326dc25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:57.860539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-055b382f-c5d3-4bf5-8244-1d9c326dc25f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:02:57.861813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-055b382f-c5d3-4bf5-8244-1d9c326dc25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:02:57.945613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 593/1202] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:02:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:02:57.970448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65315d60-016a-4b7e-b53c-cbece9f9891b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:58.102975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65315d60-016a-4b7e-b53c-cbece9f9891b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:58.143561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:58.144377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:58.144766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:58.145169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:58.158012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:58.159748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:02:58.181688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:02:58.182757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:02:58.183771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:02:58.183771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:02:58.183771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:02:58.183771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:02:58.183771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 610/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:02:58.189556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b382f-c5d3-4bf5-8244-1d9c326dc25f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:58.325571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b382f-c5d3-4bf5-8244-1d9c326dc25f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:58.367559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:58.374998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23355f53-36ea-4c01-b4e7-eb7e8f3b76b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:58.376012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23355f53-36ea-4c01-b4e7-eb7e8f3b76b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:58.376309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23355f53-36ea-4c01-b4e7-eb7e8f3b76b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:58.376587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23355f53-36ea-4c01-b4e7-eb7e8f3b76b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:58.377416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23355f53-36ea-4c01-b4e7-eb7e8f3b76b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:02:58.377722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23355f53-36ea-4c01-b4e7-eb7e8f3b76b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:58.379174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23355f53-36ea-4c01-b4e7-eb7e8f3b76b4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:58.392165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 594/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:02:58.398078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:02:58.469063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:02:58.527896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea614e42-ce98-4a3e-8e58-bd8c41e85baf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:02:58.574082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb8446-d2d7-4558-9839-2337d85846cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:02:58.574777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb8446-d2d7-4558-9839-2337d85846cc None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:02:58.575064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb8446-d2d7-4558-9839-2337d85846cc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:02:58.575342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb8446-d2d7-4558-9839-2337d85846cc None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:02:58.576110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb8446-d2d7-4558-9839-2337d85846cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:02:58.577574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb8446-d2d7-4558-9839-2337d85846cc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:02:58.600065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 611/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 07:02:58 2022] POST /identity/v3/users => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:03:00.930376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0f8b1-8ba4-44f4-8533-aaac8b97e8b0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:00.931014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0f8b1-8ba4-44f4-8533-aaac8b97e8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:00.931014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0f8b1-8ba4-44f4-8533-aaac8b97e8b0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:00.932362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 595/1206] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:00.939605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901613e1-2e4e-46bd-a5c5-6b2eb20b93eb None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:00.939605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901613e1-2e4e-46bd-a5c5-6b2eb20b93eb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:00.939605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901613e1-2e4e-46bd-a5c5-6b2eb20b93eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:00.962895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-901613e1-2e4e-46bd-a5c5-6b2eb20b93eb None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:00.962895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-901613e1-2e4e-46bd-a5c5-6b2eb20b93eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:01.050123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 612/1207] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:03:01.074476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0f8b1-8ba4-44f4-8533-aaac8b97e8b0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:01.185596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0f8b1-8ba4-44f4-8533-aaac8b97e8b0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:01.224110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:01.224110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:01.224110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:01.224110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:01.235884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:03:01.244253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:01.245390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:01.246887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 596/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:03:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:01.252153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901613e1-2e4e-46bd-a5c5-6b2eb20b93eb None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:01.302505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:01.386819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901613e1-2e4e-46bd-a5c5-6b2eb20b93eb None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:01.425878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:01.426609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:01.426904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:01.427172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:01.427913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:01.428204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:01.429659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:01.442129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 613/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:03:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:01.448361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:01.504106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:01.560138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8561-3535-48ed-9375-d639a903063f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:01.595599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:01.596405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:01.596687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:01.596958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:01.608078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:01.609566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:01.627232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:01.628237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:01.629616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:01.629616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:01.629616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:01.629616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:01.629616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 597/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:03:01.634301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:01.764976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:01.767880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bf7b0-9bd4-42e7-a79e-3e069474751f None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:01.805744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:01.806420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:01.806705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:01.806983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:01.807710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:01.808019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:01.809445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:01.821832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 614/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:01.827803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:01.942074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096cab-d1bb-4882-9da6-61ad309d21ae None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:01.943663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:01.981600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:01.982493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:01.982826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:01.983152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:01.994900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:01.997476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:03:02.018568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:02.019700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:02.020709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:02.025172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 598/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:02.027238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:02.152967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26588f37-a125-4dc2-b42b-4d91ec143a90 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:02.190728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:02.191408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:02.191697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:02.191952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:02.192808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:02.193184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:02.194664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:02.214823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 615/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 07:03:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:02.220667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:02.340100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8b643-a072-4223-8d8f-ba9a7fc5c4d1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:02.386412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:02.387744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:02.387744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:02.387923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:02.430531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:02.481077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:02.482762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:02.604237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 599/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:03:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:03:02.610942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:02.634216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:02.734422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d23fa-65ff-43ec-ba45-e0a55f32aa94 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:02.786002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:02.786820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:02.787097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:02.787363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:02.798797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:02.800308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:02.820244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:02.821685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:02.826364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:02.826364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:02.826364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:02.826364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:02.826364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 616/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:02.827925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:02.892990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:02.933400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba65e12-3d49-4372-8381-07894e5efa1f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:02.970454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:02.971153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:02.971358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:02.971634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:02.972365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:02.972677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:02.974080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:02.986309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 600/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:02.993204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:03.071045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:03.107257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e9d08-16e9-4224-9d94-b971153bda72 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:03.142446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:03.143304 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:03.143529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:03.143800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:03.150672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:03.152013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:03:03.166449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:03.167661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:03.168886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:03.173686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:03.174700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 617/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:03:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:03.300292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70da4d7-1e89-41d8-93fb-e22eb7b9dd2b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:03.334561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:03.347366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab439e-e877-449a-a6ff-49a3720f92f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:03.348193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab439e-e877-449a-a6ff-49a3720f92f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:03.348572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab439e-e877-449a-a6ff-49a3720f92f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:03.348944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab439e-e877-449a-a6ff-49a3720f92f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:03.349884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab439e-e877-449a-a6ff-49a3720f92f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:03.350286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab439e-e877-449a-a6ff-49a3720f92f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:03.351817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab439e-e877-449a-a6ff-49a3720f92f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:03.365128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 601/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:03.371188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:03.476294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4306e-d7c4-4319-b624-7829ac08d81d None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:03.507031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:03.513752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e90ef-8624-49e0-a765-1efd5d40cf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:03.516428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e90ef-8624-49e0-a765-1efd5d40cf75 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:03.516428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e90ef-8624-49e0-a765-1efd5d40cf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:03.516428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e90ef-8624-49e0-a765-1efd5d40cf75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:03.516428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e90ef-8624-49e0-a765-1efd5d40cf75 None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=60ca99d3679b4154a669661e28e8269b) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:03.516428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e90ef-8624-49e0-a765-1efd5d40cf75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:03.517598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e90ef-8624-49e0-a765-1efd5d40cf75 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:03.523879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 618/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:03:03 2022] GET /identity/v3/role_assignments?user.id=60ca99d3679b4154a669661e28e8269b&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:05.520585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac6664-5cea-4cfc-ac9f-5ad61c694039 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:05.522789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac6664-5cea-4cfc-ac9f-5ad61c694039 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:05.522789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac6664-5cea-4cfc-ac9f-5ad61c694039 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:05.522789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 602/1220] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:03:05.527792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a3861-8e14-48ba-bca6-b85e48b5ed6d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:05.528231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a3861-8e14-48ba-bca6-b85e48b5ed6d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:05.528338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a3861-8e14-48ba-bca6-b85e48b5ed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:05.545629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-209a3861-8e14-48ba-bca6-b85e48b5ed6d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:05.546804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-209a3861-8e14-48ba-bca6-b85e48b5ed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:05.619385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 619/1221] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 07:03:05.641995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac6664-5cea-4cfc-ac9f-5ad61c694039 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:05.762703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac6664-5cea-4cfc-ac9f-5ad61c694039 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:05.803387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:05.804263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:05.804263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:05.804263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:05.815669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:03:05.823655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:05.824866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:05.825929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 603/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:03:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:05.831305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a3861-8e14-48ba-bca6-b85e48b5ed6d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:05.937152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a3861-8e14-48ba-bca6-b85e48b5ed6d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:05.961419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:05.972860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:05.973565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:05.975411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:05.975411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:05.975411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] RBAC: Adding query filter params (name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:05.975411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:05.976612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:05.985234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 620/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:03:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:05.991247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:06.039063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:06.100887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3388b2-d5c7-46f6-b89e-127e29d81d4a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:06.136539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:06.137367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:06.137641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:06.137905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:06.148120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:06.149788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:06.167500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:06.168471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:06.169415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:06.169415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:06.169415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:06.169415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:06.169415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 604/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:06.174848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:06.281349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5609c54-0499-4aac-b86d-fbc45ce6247c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:06.316400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:06.317090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:06.317368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:06.317646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:06.318399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:06.318709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:06.320091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:06.332689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 621/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:06.338278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:06.433353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:06.441666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1082f-5ef8-49e4-801d-0d4559deaa59 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:06.479325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:06.480136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:06.480420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:06.480693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:06.491111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:06.492584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:03:06.508054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:06.509124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:06.510525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 605/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:06.517425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:06.565762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:06.620689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8399d-5092-4d48-8362-ef081e29d749 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:06.654800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:06.655498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:06.655792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:06.656092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:06.656941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:06.657295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:06.658743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:06.678114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 622/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 07:03:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:06.683848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:06.787379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79cd00-6a97-4360-8380-12dd02fded4d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:06.822981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:06.823784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:06.824068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:06.824339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:06.835319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:06.836997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:06.855552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:06.856652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:06.857679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:06.857679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:06.857679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:06.857679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:06.857679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 606/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:06.862759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:06.978641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec5bcf-2580-4cb3-9a3a-baf5f524b004 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:06.980963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:07.021867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:07.022510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:07.022790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:07.023109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:07.023828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:07.024178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:07.025573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:07.043425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 623/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:07.044914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:07.055383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:07.156820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a82d-5fe7-432e-bc16-5fe3a72f7b3a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:07.197268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:07.198176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:07.198568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:07.198904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:07.209368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:07.211281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:03:07.230493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:07.231564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:07.232494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:07.232937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 607/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:07.239092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:07.354252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ca5fe-0cdc-44e5-9de9-716b96b955ff None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:07.390382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69d99b-027e-47b4-bf46-5faf04a15d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:07.391128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69d99b-027e-47b4-bf46-5faf04a15d2f None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:07.391404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69d99b-027e-47b4-bf46-5faf04a15d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:07.391669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69d99b-027e-47b4-bf46-5faf04a15d2f None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:07.392450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69d99b-027e-47b4-bf46-5faf04a15d2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:07.392764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69d99b-027e-47b4-bf46-5faf04a15d2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:07.394226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69d99b-027e-47b4-bf46-5faf04a15d2f None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:07.406479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 624/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:07.413561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:07.489134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:07.528987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5be490-a01d-4a8b-90a8-8b355a269e75 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:07.570161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16565cd6-b65b-4c8d-a211-f8a09ecf6e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:07.571479 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565cd6-b65b-4c8d-a211-f8a09ecf6e11 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:07.571841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565cd6-b65b-4c8d-a211-f8a09ecf6e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:07.572109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565cd6-b65b-4c8d-a211-f8a09ecf6e11 None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/60ca99d3679b4154a669661e28e8269b/roles/59c2d2f6ca754e289f3766b15292756e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:07.593733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16565cd6-b65b-4c8d-a211-f8a09ecf6e11 None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=60ca99d3679b4154a669661e28e8269b, role_id=59c2d2f6ca754e289f3766b15292756e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:07.595595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16565cd6-b65b-4c8d-a211-f8a09ecf6e11 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:07.608253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 608/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:03:07 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/60ca99d3679b4154a669661e28e8269b/roles/59c2d2f6ca754e289f3766b15292756e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:03:09.832760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfaaed-2921-4996-8e00-b3653e779f49 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:09.833352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfaaed-2921-4996-8e00-b3653e779f49 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:09.833352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfaaed-2921-4996-8e00-b3653e779f49 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:09.834730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 625/1233] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:09.840555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eac17-7ea1-4cac-9bc0-43f5b45da26f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:09.841019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eac17-7ea1-4cac-9bc0-43f5b45da26f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:09.841252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eac17-7ea1-4cac-9bc0-43f5b45da26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:09.858739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d48eac17-7ea1-4cac-9bc0-43f5b45da26f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:09.860032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d48eac17-7ea1-4cac-9bc0-43f5b45da26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:09.937795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 609/1234] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:09.966050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfaaed-2921-4996-8e00-b3653e779f49 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:09.995782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:10.079431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfaaed-2921-4996-8e00-b3653e779f49 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:10.117219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:10.118022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:10.118321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:10.118564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:10.132999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:03:10.142307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:10.143488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 07:03:10.144571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:10.151116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 626/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 07:03:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:03:10.152580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eac17-7ea1-4cac-9bc0-43f5b45da26f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:10.211291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:10.283457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eac17-7ea1-4cac-9bc0-43f5b45da26f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:10.322474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:10.323148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:10.323429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:10.323756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:10.324504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:10.324810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:10.326224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:10.334819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 610/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 07:03:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:10.340656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:10.431572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:10.448529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296b72b-edc2-4a26-aeb8-86cbcd7a3d9c None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:10.483949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:10.484849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:10.485290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:10.485562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:10.496355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:10.497831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:10.514603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:10.515861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:10.517315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:10.517315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:10.517315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:10.517315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:10.521082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 627/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:10.521977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:10.626539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e53f-d749-436d-8277-626a03edf7d9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:10.635514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:10.661644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:10.662312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:10.662591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:10.662864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:10.663581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:10.663886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:10.665289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:10.678834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 611/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:10.684511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:10.814176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd686b-ddd7-42b2-aa33-f8fa6b3f42ea None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:10.862642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:10.863628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:10.863781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:10.864025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:10.867217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:10.874339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:10.877483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:03:10.895894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:10.897175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:10.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 628/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:10.903314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:11.021590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962ef01-d072-45a4-9094-ecadbc96dba0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:11.056396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:11.057148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:11.057431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:11.057705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:11.058621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:11.058936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:11.060490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:11.079863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 612/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 07:03:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:11.085850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:11.191380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dca33-21ca-479c-8a13-7f8fa691ba31 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:11.227014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:11.227676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:11.228009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:11.228281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:11.298037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:11.305261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:11.307052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:11.412327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 629/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:03:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:03:11.413714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:11.442232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:11.521014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c590fc2-6611-488a-b48f-bc6f19727827 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:11.557075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:11.557813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:11.558092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:11.558363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:11.568651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:11.570219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:11.589456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:11.590590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:11.591526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:11.591526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:11.591526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:11.591526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:11.591526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 613/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:11.596587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:11.703815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:11.710444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68426927-1ca7-4971-8606-4c53e1090853 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:11.747361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:11.748058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:11.748325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:11.748597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:11.749368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:11.749675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:11.751213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:11.763477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 630/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:11.769879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:11.858743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:11.884452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aa108-6706-48aa-963d-52cb35ddaea3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:11.919818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:11.920653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:11.920936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:11.921295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:11.928654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:11.930127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:03:11.943757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:11.944841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:11.946290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 614/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:03:11.952256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:12.065830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b54d1-66ab-4962-bcf8-9603d5e691a4 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:12.106799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fbc14-008f-4570-8f36-c74f4a767879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:12.107548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbc14-008f-4570-8f36-c74f4a767879 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:12.107865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbc14-008f-4570-8f36-c74f4a767879 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:12.108172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbc14-008f-4570-8f36-c74f4a767879 None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:12.109310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fbc14-008f-4570-8f36-c74f4a767879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:12.109687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fbc14-008f-4570-8f36-c74f4a767879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:12.111468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fbc14-008f-4570-8f36-c74f4a767879 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:12.125962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 631/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:12.133088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:12.254862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf732c-8feb-46ef-ade2-63d654694875 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:12.292405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7b030-3705-4aa2-8965-26d45dfe2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:12.293096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7b030-3705-4aa2-8965-26d45dfe2570 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:12.293381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7b030-3705-4aa2-8965-26d45dfe2570 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:12.293682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7b030-3705-4aa2-8965-26d45dfe2570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:12.294492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7b030-3705-4aa2-8965-26d45dfe2570 None admin] RBAC: Adding query filter params (role.id=59c2d2f6ca754e289f3766b15292756e, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=60ca99d3679b4154a669661e28e8269b) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:12.294804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7b030-3705-4aa2-8965-26d45dfe2570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:12.296246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7b030-3705-4aa2-8965-26d45dfe2570 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:12.302411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 615/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:03:12 2022] GET /identity/v3/role_assignments?user.id=60ca99d3679b4154a669661e28e8269b&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=59c2d2f6ca754e289f3766b15292756e => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:14.428202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a889e0-76a7-475c-9d7f-ddcada89b84b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:14.428738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a889e0-76a7-475c-9d7f-ddcada89b84b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:14.428738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a889e0-76a7-475c-9d7f-ddcada89b84b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:14.430090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 632/1247] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:14.435647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089108a-f230-431a-bb0c-8643cffc5c9a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:14.436124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089108a-f230-431a-bb0c-8643cffc5c9a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:14.436219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089108a-f230-431a-bb0c-8643cffc5c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:14.441885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:14.455623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5089108a-f230-431a-bb0c-8643cffc5c9a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:14.456841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5089108a-f230-431a-bb0c-8643cffc5c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:14.531932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 616/1248] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:14.555853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a889e0-76a7-475c-9d7f-ddcada89b84b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:14.576328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:14.667900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a889e0-76a7-475c-9d7f-ddcada89b84b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:14.706650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:14.707457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:14.707734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:14.708011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:14.719973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:03:14.727723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:14.728804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:14.729734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 633/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:03:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:14.736042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089108a-f230-431a-bb0c-8643cffc5c9a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:14.870656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089108a-f230-431a-bb0c-8643cffc5c9a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:14.904560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:14.910541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:14.911224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:14.911510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:14.911800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:14.912573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:14.912877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:14.914397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:14.925638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 617/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:03:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:03:14.931735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:15.026054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:15.053827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffb12b-e261-422b-a0db-4f82fc169cb8 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:15.092143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:15.092965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:15.093273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:15.093545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:15.105789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:15.107245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:15.127162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:15.128327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:15.129856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:15.129856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:15.129856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:15.129856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:15.129856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 634/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:15.134750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:15.253670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d4563-828e-4b93-ade0-a9be2f3ca89c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:15.291713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:15.292412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:15.292741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:15.293079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:15.293807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:15.294105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:15.295523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:15.308698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 618/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:15.315875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:15.429120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf846b0f-3c70-4316-b8f9-0f3d8ac1122b None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:15.459742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:15.470898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:15.471823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:15.472121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:15.472565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:15.481566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:15.483007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:03:15.499761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:15.500823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:15.501935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 635/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:15.507680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:15.581400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:15.633451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6eb0-3c0b-47c5-8976-10aac111d54f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:15.674839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:15.675758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:15.676038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:15.676313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:15.677200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:15.677540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:15.678962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:15.699793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 619/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 07:03:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:15.709358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:15.842140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b891c45-db42-4cf0-8e7e-1672ca6354b5 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:15.884644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:15.885344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:15.885656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:15.885947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:15.960188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:15.976858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:15.978788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:16.118775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 636/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:03:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:03:16.128413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:16.263709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b269cd-093c-4954-997a-98630029d838 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:16.279374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:16.311242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:16.312137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:16.312459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:16.313175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:16.327190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:16.329349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:16.349607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:16.350943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:16.352129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:16.352129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:16.352129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:16.352129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:16.352129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 620/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:03:16.357241 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:16.465270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:16.479626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484f401-29ac-477d-a6ae-7f798d4e1fdc None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:16.527315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:16.528120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:16.528389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:16.528732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:16.529510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:16.529875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:16.531753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:16.549029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 637/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:03:16.555196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:16.676813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367583-42ed-4907-9fda-4389ded9c44d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:16.722281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:16.723005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:16.723270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:16.723617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:16.731904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:16.733875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:03:16.751451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:16.752822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:16.753960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 621/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:16.758730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:16.881367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846b241-c131-4eb3-9f2c-e3dafcc732da None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:16.926801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08adaf7-7f80-4396-8277-7757fb1f57ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:16.927651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08adaf7-7f80-4396-8277-7757fb1f57ff None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:16.928005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08adaf7-7f80-4396-8277-7757fb1f57ff None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:16.928375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08adaf7-7f80-4396-8277-7757fb1f57ff None admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:16.929344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08adaf7-7f80-4396-8277-7757fb1f57ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:16.929734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08adaf7-7f80-4396-8277-7757fb1f57ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:16.931453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08adaf7-7f80-4396-8277-7757fb1f57ff None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:16.947714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 638/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:16.955142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:16.987710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:17.087968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33154601-5c9d-49d0-8b28-22bc288ea67d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:17.127469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1201b-77ea-4901-bc10-1864bb2fbc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:17.128119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1201b-77ea-4901-bc10-1864bb2fbc38 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:17.128405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1201b-77ea-4901-bc10-1864bb2fbc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:17.128680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1201b-77ea-4901-bc10-1864bb2fbc38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:17.129558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1201b-77ea-4901-bc10-1864bb2fbc38 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=60ca99d3679b4154a669661e28e8269b) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:17.129873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1201b-77ea-4901-bc10-1864bb2fbc38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:17.131310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1201b-77ea-4901-bc10-1864bb2fbc38 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:17.138129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 622/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:03:16 2022] GET /identity/v3/role_assignments?user.id=60ca99d3679b4154a669661e28e8269b&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=aaca343d2e4a43c88674834f08499f15 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:03:19.219699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d4d41-8cec-4f08-84ad-dae0e0c38459 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:19.220232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d4d41-8cec-4f08-84ad-dae0e0c38459 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:19.220232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d4d41-8cec-4f08-84ad-dae0e0c38459 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:19.222381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 639/1261] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:19.228700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c60758-0c44-4115-b25a-c26af2062143 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:19.229136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c60758-0c44-4115-b25a-c26af2062143 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:19.229309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c60758-0c44-4115-b25a-c26af2062143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:19.248516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c9c60758-0c44-4115-b25a-c26af2062143 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:19.249667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c9c60758-0c44-4115-b25a-c26af2062143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:19.318106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 623/1262] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:19.345439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d4d41-8cec-4f08-84ad-dae0e0c38459 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:19.357106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:19.451889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d4d41-8cec-4f08-84ad-dae0e0c38459 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:19.487103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:19.487897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:19.488175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:19.488443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:19.499140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:03:19.506747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:19.507824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:19.508842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 640/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:03:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:19.517201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c60758-0c44-4115-b25a-c26af2062143 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:19.538363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:19.631711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c60758-0c44-4115-b25a-c26af2062143 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:19.667183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:19.667840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:19.668123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:19.668394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:19.669159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:19.669468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:19.670832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:19.679142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 624/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:03:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:19.687796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:19.763615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:19.804830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd8c40-443c-49ed-afd7-4452fbfec365 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:19.848736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:19.849565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:19.849846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:19.850118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:19.861181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:19.862641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:19.880389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:19.881527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:19.882462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:19.882462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:19.882462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:19.882462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:19.882462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 641/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:03:19.890523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:19.978676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:20.003481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cf339-df63-4dd1-b12b-0ed5a84797a3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:20.038338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:20.038932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:20.039211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:20.039492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:20.040218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:20.040530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:20.041930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:20.053997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 625/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:20.060793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:20.174317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0950e4-200b-4c65-b9b8-5825d7ae4daf None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:20.194426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:20.213783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:20.214996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:20.215400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:20.215788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:20.228616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:20.230943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:03:20.247140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:20.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:20.249672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 642/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:20.254473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:20.358180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783635a9-ea77-4113-b908-d7b1ef07ebb7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:20.393482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:20.394148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:20.394431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:20.394706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:20.395544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:20.395858 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:20.397308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:20.406027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:20.415468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 626/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 07:03:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:20.421846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:20.530702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61af9e9-5403-4191-a8be-13641b151669 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:20.565134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:20.565933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:20.566209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:20.566474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:20.577607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:20.579082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:20.596182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:20.597263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:20.598535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:20.598535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:20.598535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:20.598535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:20.598535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 643/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:20.603286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:20.713226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50c244-82d4-4a20-aaa0-a660ced14b04 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:20.755222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:20.756409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:20.757111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:20.757634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:20.759009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:20.759652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:20.762613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:20.778521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 627/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:20.786737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:20.824293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:20.896767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75768-4030-4f79-a9aa-c2a1148aa6ea None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:20.931275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:20.932077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:20.932376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:20.932636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:20.939476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:20.940921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:03:20.955546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:20.956567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:20.957807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 644/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:03:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:20.962894 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:21.026799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:21.070642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea36c9-ae0e-4962-8b49-516bfbeb78b9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:21.107931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef4e2d-2791-49cf-af3c-b8b1a8a0a542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:21.108585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef4e2d-2791-49cf-af3c-b8b1a8a0a542 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:21.108868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef4e2d-2791-49cf-af3c-b8b1a8a0a542 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:21.109178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef4e2d-2791-49cf-af3c-b8b1a8a0a542 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:21.110016 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef4e2d-2791-49cf-af3c-b8b1a8a0a542 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:21.110326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef4e2d-2791-49cf-af3c-b8b1a8a0a542 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:21.111754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef4e2d-2791-49cf-af3c-b8b1a8a0a542 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:21.124591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 628/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:03:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:21.130975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:21.225913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:21.239858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecc46-bb72-4be4-bac3-00e765126efe None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:21.280200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441133-68bb-4b27-9f59-b8332ac99a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:21.281028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441133-68bb-4b27-9f59-b8332ac99a5b None admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:21.281415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441133-68bb-4b27-9f59-b8332ac99a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:21.281762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441133-68bb-4b27-9f59-b8332ac99a5b None admin] PATH_INFO: `/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/60ca99d3679b4154a669661e28e8269b/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:21.283716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441133-68bb-4b27-9f59-b8332ac99a5b None admin] RBAC: Authorizing `identity:create_grant(project_id=f18dfebf66b2442d8a63b5dd4f648f69, user_id=60ca99d3679b4154a669661e28e8269b, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:21.285172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441133-68bb-4b27-9f59-b8332ac99a5b None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:21.304734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 645/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 07:03:21 2022] PUT /identity/v3/projects/f18dfebf66b2442d8a63b5dd4f648f69/users/60ca99d3679b4154a669661e28e8269b/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:03:23.268902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f08519-e738-4891-8114-ff7176208d5d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:23.269497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f08519-e738-4891-8114-ff7176208d5d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:23.269497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f08519-e738-4891-8114-ff7176208d5d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:23.270768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 629/1274] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:23.277114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93949-bea2-4a21-8a49-17f66d01179b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:23.277805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93949-bea2-4a21-8a49-17f66d01179b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:23.278118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93949-bea2-4a21-8a49-17f66d01179b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:23.297991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-36e93949-bea2-4a21-8a49-17f66d01179b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:23.299175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-36e93949-bea2-4a21-8a49-17f66d01179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:23.368264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 646/1275] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:23.392067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f08519-e738-4891-8114-ff7176208d5d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:23.502728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f08519-e738-4891-8114-ff7176208d5d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:23.520218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:23.539323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:23.540121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:23.540403 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:23.540682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:23.551695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 07:03:23.559271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:23.560349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:23.561487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 630/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 07:03:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:23.567661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e93949-bea2-4a21-8a49-17f66d01179b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:23.653320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:23.674243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e93949-bea2-4a21-8a49-17f66d01179b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:23.721608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:23.722279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:23.722557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:23.722825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:23.723569 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:23.723871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:23.725270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:23.734060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 647/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 07:03:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:23.741003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:23.857409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ca18-1220-4839-a5a2-0907ecb3d533 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:23.892405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:23.893241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:23.893520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:23.893786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:23.906311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:23.907752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:23.926392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:23.927347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:23.928318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:23.928318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:23.928318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:23.928318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:23.928318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 631/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:23.933589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:24.048725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85531035-cf64-4a39-b760-9b9bac948201 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:24.090456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd585-7a96-4874-883d-843559239018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:24.091110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd585-7a96-4874-883d-843559239018 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:24.091393 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd585-7a96-4874-883d-843559239018 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:24.091666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd585-7a96-4874-883d-843559239018 None admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:24.092378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd585-7a96-4874-883d-843559239018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:24.092677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd585-7a96-4874-883d-843559239018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:24.094122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd585-7a96-4874-883d-843559239018 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:24.105922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 648/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:24.113256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:24.146027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:24.222678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453810c-f1dd-4c07-875d-dec4cdb1f6bf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:24.262268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:24.263071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:24.263352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:24.263646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:24.272495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:24.273974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 07:03:24.288488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:24.289636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:24.290635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 632/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:24.296378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd585-7a96-4874-883d-843559239018 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:24.302618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:24.425180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd585-7a96-4874-883d-843559239018 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:24.466267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:24.466963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:24.467247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:24.467520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:24.468359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:24.468831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:24.470361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:24.487690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 649/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 07:03:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:24.493698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:24.597071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:24.608167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67c73-695e-4eb9-ac73-ce2683332eb9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:24.642957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:24.643585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:24.643864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:24.644135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:24.729660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:24.731140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:24.833663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:24.844226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 633/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 07:03:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 07:03:24.851074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:24.918301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:24.958668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d96061-b6d9-421d-9f5b-29b89d9f8798 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:24.994097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:24.994909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:24.995193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:24.995462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:25.007400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:25.008865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 07:03:25.027001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:25.028056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:25.029167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 07:03:25.029167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 07:03:25.029167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 07:03:25.029167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:25.029167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 650/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:25.034089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:25.137477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040e406-345b-4899-a411-5e697e3fe9fe None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:25.173665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:25.174321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:25.174606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:25.174880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:25.175607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:25.175904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:25.177305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:25.188841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 634/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 07:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:25.198214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:25.308803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b04e61-e603-4e26-9e3f-60e1848a5169 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:25.323355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:25.345511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:25.346309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:25.346590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:25.346862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:25.353656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:25.355121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 07:03:25.368769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:25.369853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: ERROR keystone.server.flask.application Jul 12 07:03:25.371153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 651/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 07:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:25.377830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:25.441426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:25.493439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6eb98-f81c-4602-880e-788d8083c31b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:25.528097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c5a74-a7b9-4ead-8d5f-931f128ac048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:25.528746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c5a74-a7b9-4ead-8d5f-931f128ac048 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:25.529025 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c5a74-a7b9-4ead-8d5f-931f128ac048 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:25.529324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c5a74-a7b9-4ead-8d5f-931f128ac048 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:25.530107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c5a74-a7b9-4ead-8d5f-931f128ac048 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:25.530417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c5a74-a7b9-4ead-8d5f-931f128ac048 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:25.531868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c5a74-a7b9-4ead-8d5f-931f128ac048 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:25.543804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 635/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 07:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:25.550929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:25.667731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca030ca-9d32-4ac5-88b9-cfa6cb58d2b2 None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:25.703258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89938a55-13e1-4179-9211-44478e83f5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:25.703916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89938a55-13e1-4179-9211-44478e83f5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:25.704193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89938a55-13e1-4179-9211-44478e83f5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:25.704468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89938a55-13e1-4179-9211-44478e83f5b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:25.705318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89938a55-13e1-4179-9211-44478e83f5b6 None admin] RBAC: Adding query filter params (role.id=aaca343d2e4a43c88674834f08499f15, scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69, user.id=60ca99d3679b4154a669661e28e8269b) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:25.705620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89938a55-13e1-4179-9211-44478e83f5b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:25.707053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89938a55-13e1-4179-9211-44478e83f5b6 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:25.713122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 652/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 07:03:25 2022] GET /identity/v3/role_assignments?user.id=60ca99d3679b4154a669661e28e8269b&scope.project.id=f18dfebf66b2442d8a63b5dd4f648f69&role.id=aaca343d2e4a43c88674834f08499f15 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:27.743945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec66c-7881-4b11-b577-c10ae5cc31ca None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:27.744542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec66c-7881-4b11-b577-c10ae5cc31ca None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:27.744542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec66c-7881-4b11-b577-c10ae5cc31ca None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:27.746302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 636/1288] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:27.752626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40f6f5-ba2e-479e-adfd-5f7948377a17 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:27.753234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40f6f5-ba2e-479e-adfd-5f7948377a17 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:27.753332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40f6f5-ba2e-479e-adfd-5f7948377a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:27.780246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4c40f6f5-ba2e-479e-adfd-5f7948377a17 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:27.781910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4c40f6f5-ba2e-479e-adfd-5f7948377a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:27.814495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:27.853542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 653/1289] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:27.876124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ec66c-7881-4b11-b577-c10ae5cc31ca None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:27.983005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ec66c-7881-4b11-b577-c10ae5cc31ca None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:27.994892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:28.027899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:28.028717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:28.028994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:28.029302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:28.029952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:28.031358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:28.043439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:28.044680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:28.045849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 637/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:03:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 07:03:28.050947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40f6f5-ba2e-479e-adfd-5f7948377a17 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:28.158849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40f6f5-ba2e-479e-adfd-5f7948377a17 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:28.194428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027be62-28b8-4d57-8fa2-5c2b54c451b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:28.195443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027be62-28b8-4d57-8fa2-5c2b54c451b0 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:28.195722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027be62-28b8-4d57-8fa2-5c2b54c451b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:28.195989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027be62-28b8-4d57-8fa2-5c2b54c451b0 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:28.196692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027be62-28b8-4d57-8fa2-5c2b54c451b0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:28.197002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027be62-28b8-4d57-8fa2-5c2b54c451b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:28.198400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027be62-28b8-4d57-8fa2-5c2b54c451b0 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:28.203362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 654/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 07:03:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:28.212533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:28.318163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa2463-901b-4472-b9fe-c650c0dfb186 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:28.353457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8200953-681a-4050-8d1b-7b44c11c4f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:28.354099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8200953-681a-4050-8d1b-7b44c11c4f17 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:28.354387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8200953-681a-4050-8d1b-7b44c11c4f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:28.354651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8200953-681a-4050-8d1b-7b44c11c4f17 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:28.355345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8200953-681a-4050-8d1b-7b44c11c4f17 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:28.355642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8200953-681a-4050-8d1b-7b44c11c4f17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:28.357000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8200953-681a-4050-8d1b-7b44c11c4f17 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:28.361811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 638/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 07:03:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:30.359131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc9789-f3b5-4165-bba1-3adce7231791 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:30.359973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc9789-f3b5-4165-bba1-3adce7231791 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:30.359973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc9789-f3b5-4165-bba1-3adce7231791 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:30.362099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 655/1293] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:30.367557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23340def-fa39-4039-a408-c053ef8792da None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:30.367990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23340def-fa39-4039-a408-c053ef8792da None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:30.368097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23340def-fa39-4039-a408-c053ef8792da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:30.386690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-23340def-fa39-4039-a408-c053ef8792da None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:30.387882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-23340def-fa39-4039-a408-c053ef8792da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:30.418108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:30.458652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 639/1294] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:30.482487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc9789-f3b5-4165-bba1-3adce7231791 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:30.531390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:30.605808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc9789-f3b5-4165-bba1-3adce7231791 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:30.641469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2e59c-aeeb-4fcf-9cdb-f2608f7d427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:30.642048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e59c-aeeb-4fcf-9cdb-f2608f7d427a None admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:30.642326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e59c-aeeb-4fcf-9cdb-f2608f7d427a None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:30.642597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e59c-aeeb-4fcf-9cdb-f2608f7d427a None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:30.643311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2e59c-aeeb-4fcf-9cdb-f2608f7d427a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:30.644678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2e59c-aeeb-4fcf-9cdb-f2608f7d427a None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:30.661602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 656/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 07:03:30 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:03:32.755232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d20478-a339-4968-a64c-408ef0ecba99 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:32.755820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d20478-a339-4968-a64c-408ef0ecba99 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:32.755820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d20478-a339-4968-a64c-408ef0ecba99 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:32.757559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 640/1296] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:32.766790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d64d5-5bc3-4b1d-9d67-e2f637c206fb None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:32.767439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d64d5-5bc3-4b1d-9d67-e2f637c206fb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:32.767662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d64d5-5bc3-4b1d-9d67-e2f637c206fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:32.789662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-403d64d5-5bc3-4b1d-9d67-e2f637c206fb None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:32.790862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-403d64d5-5bc3-4b1d-9d67-e2f637c206fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:32.870035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 657/1297] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:32.893893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d20478-a339-4968-a64c-408ef0ecba99 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:33.024901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d20478-a339-4968-a64c-408ef0ecba99 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:33.060326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:33.061116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:33.061427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:33.061694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:33.062533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:33.063905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:33.076289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:33.081196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:33.082304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 641/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:03:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:33.085933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d64d5-5bc3-4b1d-9d67-e2f637c206fb None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:33.113335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:33.197678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d64d5-5bc3-4b1d-9d67-e2f637c206fb None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:33.232762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:33.233449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:33.233731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:33.234024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:33.234752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] RBAC: Adding query filter params (name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:33.235062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:33.236460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:33.241744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 658/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 07:03:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:33.248464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:33.288996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:33.365184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1514e-9306-4f5c-9774-44c2cbfd51f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:33.402634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246a68e-57c5-44d8-a9a0-e417356251d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:33.403286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a68e-57c5-44d8-a9a0-e417356251d2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:33.403569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a68e-57c5-44d8-a9a0-e417356251d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:33.403846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a68e-57c5-44d8-a9a0-e417356251d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:33.404635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246a68e-57c5-44d8-a9a0-e417356251d2 None admin] RBAC: Adding query filter params (interface=public, service_id=45148a2902dd4be1b0e9e11f5805c7a3, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:33.404944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246a68e-57c5-44d8-a9a0-e417356251d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:33.406503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246a68e-57c5-44d8-a9a0-e417356251d2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:33.412292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 642/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 07:03:33 2022] GET /identity/v3/endpoints?service_id=45148a2902dd4be1b0e9e11f5805c7a3&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:33.418183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:33.504670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:33.527292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736672fe-ff05-45ae-98a9-51f1cc236aff None admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:33.562050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915120a-e129-4d05-8731-e6234d1581b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:33.562792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915120a-e129-4d05-8731-e6234d1581b4 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:33.563073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915120a-e129-4d05-8731-e6234d1581b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:33.563343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915120a-e129-4d05-8731-e6234d1581b4 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:33.564017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915120a-e129-4d05-8731-e6234d1581b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:33.565418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915120a-e129-4d05-8731-e6234d1581b4 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:33.572725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 659/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 07:03:33 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:03:35.560565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bde24-d04b-4df7-8f1d-8948c4e8f723 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:35.561503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bde24-d04b-4df7-8f1d-8948c4e8f723 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:35.561659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bde24-d04b-4df7-8f1d-8948c4e8f723 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:35.564253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 643/1302] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:03:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:03:35.570824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bde1d-b417-48f2-94b4-9f3a9bc564a9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:35.571521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bde1d-b417-48f2-94b4-9f3a9bc564a9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:35.571695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bde1d-b417-48f2-94b4-9f3a9bc564a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:35.591815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bc3bde1d-b417-48f2-94b4-9f3a9bc564a9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:03:35.593171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bc3bde1d-b417-48f2-94b4-9f3a9bc564a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:03:35.666014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 660/1303] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:03:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 07:03:35.694931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bde24-d04b-4df7-8f1d-8948c4e8f723 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:35.804139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:35.814372 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bde24-d04b-4df7-8f1d-8948c4e8f723 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:35.850202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:35.850994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:35.851276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:35.851549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:35.852198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:35.853614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 07:03:35.865748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 07:03:35.866910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 12 07:03:35.867964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 644/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 07:03:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 07:03:35.873280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bde1d-b417-48f2-94b4-9f3a9bc564a9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:35.956987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:35.982344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bde1d-b417-48f2-94b4-9f3a9bc564a9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:36.017400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb6471-7c93-4f95-a4ed-39cd4106f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:36.018056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6471-7c93-4f95-a4ed-39cd4106f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:36.018332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6471-7c93-4f95-a4ed-39cd4106f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:36.018599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6471-7c93-4f95-a4ed-39cd4106f0f2 None admin] PATH_INFO: `/v3/services` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:36.019296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb6471-7c93-4f95-a4ed-39cd4106f0f2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:03:36.019596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb6471-7c93-4f95-a4ed-39cd4106f0f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:36.020955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb6471-7c93-4f95-a4ed-39cd4106f0f2 None admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:36.026318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 661/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 07:03:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:03:36.032259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:03:36.138842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0282a63-40fb-4adf-999d-d411e520c118 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:03:36.173277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab857c-426f-4148-a4da-a58c7458f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:03:36.173911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab857c-426f-4148-a4da-a58c7458f7a6 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:03:36.174189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab857c-426f-4148-a4da-a58c7458f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:03:36.174460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab857c-426f-4148-a4da-a58c7458f7a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:03:36.175189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab857c-426f-4148-a4da-a58c7458f7a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:03:36.176551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab857c-426f-4148-a4da-a58c7458f7a6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:03:36.198158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 645/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 07:03:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:03:46.023114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=30 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:03:46.193030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:39.202001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d98798-0a03-43cd-93af-a54453c07379 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:39.202699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d98798-0a03-43cd-93af-a54453c07379 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:39.202699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d98798-0a03-43cd-93af-a54453c07379 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:39.209863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db01a8b-ffbd-4bf6-ba9a-110a2b89423c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:39.210394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db01a8b-ffbd-4bf6-ba9a-110a2b89423c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:39.210394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db01a8b-ffbd-4bf6-ba9a-110a2b89423c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:39.212324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 662/1307] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:04:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:39.292610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1db01a8b-ffbd-4bf6-ba9a-110a2b89423c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:39.293781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1db01a8b-ffbd-4bf6-ba9a-110a2b89423c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:39.410643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 646/1308] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:04:39.461850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8984c-df7e-457b-8c5c-32ee7e92281c None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:39.462034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8984c-df7e-457b-8c5c-32ee7e92281c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:39.462339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8984c-df7e-457b-8c5c-32ee7e92281c None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:39.464198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 663/1309] 10.222.0.25 () {54 vars in 964 bytes} [Tue Jul 12 07:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:39.472090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547dc74-c98a-4dae-9ddc-6b0ca0d3c76e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:39.472389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547dc74-c98a-4dae-9ddc-6b0ca0d3c76e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:39.472670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547dc74-c98a-4dae-9ddc-6b0ca0d3c76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:39.550630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d547dc74-c98a-4dae-9ddc-6b0ca0d3c76e None None] MFA Rules not processed for user `754c137a554941d99f7ef9c380a55ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:39.551870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d547dc74-c98a-4dae-9ddc-6b0ca0d3c76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:39.573462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:39.648801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 647/1310] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Jul 12 07:04:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:04:39.660023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8984c-df7e-457b-8c5c-32ee7e92281c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:04:39.661853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b0f8984c-df7e-457b-8c5c-32ee7e92281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:39.839875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8984c-df7e-457b-8c5c-32ee7e92281c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:04:39.918521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3ccc4-22d4-45fa-981a-dcc38f7c1a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:04:39.919698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3ccc4-22d4-45fa-981a-dcc38f7c1a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:39.920186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3ccc4-22d4-45fa-981a-dcc38f7c1a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:39.920697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3ccc4-22d4-45fa-981a-dcc38f7c1a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:39.922780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b7b3ccc4-22d4-45fa-981a-dcc38f7c1a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:39.981404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:40.050507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3ccc4-22d4-45fa-981a-dcc38f7c1a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:04:40.052948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3ccc4-22d4-45fa-981a-dcc38f7c1a86 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:04:40.079885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 664/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:04:42.384983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f437363-3abb-4e53-8d1b-07878c49cea7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:42.385587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f437363-3abb-4e53-8d1b-07878c49cea7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:42.385587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f437363-3abb-4e53-8d1b-07878c49cea7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:42.387019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 648/1312] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:42.392388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576db5e8-9926-46d7-9d72-07167696a2bf None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:42.392846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576db5e8-9926-46d7-9d72-07167696a2bf None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:42.392939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576db5e8-9926-46d7-9d72-07167696a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:42.466497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-576db5e8-9926-46d7-9d72-07167696a2bf None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:42.468082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-576db5e8-9926-46d7-9d72-07167696a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:42.572400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 665/1313] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:04:42.611547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f437363-3abb-4e53-8d1b-07878c49cea7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:04:42.613313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f437363-3abb-4e53-8d1b-07878c49cea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:42.726388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f437363-3abb-4e53-8d1b-07878c49cea7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:04:42.824264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:42.828253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d32d0-3efa-41b0-baf8-8f6df60a2fd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:04:42.829137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d32d0-3efa-41b0-baf8-8f6df60a2fd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:42.829519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d32d0-3efa-41b0-baf8-8f6df60a2fd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:42.829897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d32d0-3efa-41b0-baf8-8f6df60a2fd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:42.832266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2c9d32d0-3efa-41b0-baf8-8f6df60a2fd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:42.962521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d32d0-3efa-41b0-baf8-8f6df60a2fd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:04:42.964141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d32d0-3efa-41b0-baf8-8f6df60a2fd8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:04:42.984158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 649/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:04:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:04:45.336261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d5331-9341-47ac-8efc-4bd9bbe0c662 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:45.337147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d5331-9341-47ac-8efc-4bd9bbe0c662 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:45.337147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d5331-9341-47ac-8efc-4bd9bbe0c662 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:45.339245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 666/1315] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:04:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:45.344832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839543f-c9c2-442d-a6a2-09abb2654dd4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:45.345301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839543f-c9c2-442d-a6a2-09abb2654dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:45.345394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839543f-c9c2-442d-a6a2-09abb2654dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:45.414561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f839543f-c9c2-442d-a6a2-09abb2654dd4 None None] MFA Rules not processed for user `4b27ececfd83412c9c497073beaf72f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:45.415771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f839543f-c9c2-442d-a6a2-09abb2654dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:45.478879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:45.523911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 650/1316] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:04:45.579216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687860e4-fcb4-4e8e-9818-0e88bc71c188 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:45.579547 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687860e4-fcb4-4e8e-9818-0e88bc71c188 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:45.579966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687860e4-fcb4-4e8e-9818-0e88bc71c188 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:45.582235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 667/1317] 10.222.0.25 () {54 vars in 964 bytes} [Tue Jul 12 07:04:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:45.592015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20e145-e856-4ce3-a1c6-7f1c3468d2d7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:45.592327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20e145-e856-4ce3-a1c6-7f1c3468d2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:45.592769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20e145-e856-4ce3-a1c6-7f1c3468d2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:45.681294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7f20e145-e856-4ce3-a1c6-7f1c3468d2d7 None None] MFA Rules not processed for user `754c137a554941d99f7ef9c380a55ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:45.682517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7f20e145-e856-4ce3-a1c6-7f1c3468d2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:45.764039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 651/1318] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Jul 12 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:04:45.773452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687860e4-fcb4-4e8e-9818-0e88bc71c188 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:04:45.775025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-687860e4-fcb4-4e8e-9818-0e88bc71c188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:45.785954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:45.895776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687860e4-fcb4-4e8e-9818-0e88bc71c188 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:04:45.982501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c342d-6767-4b71-81d4-501b9164cfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:04:45.983171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c342d-6767-4b71-81d4-501b9164cfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:45.983524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c342d-6767-4b71-81d4-501b9164cfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:45.983821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c342d-6767-4b71-81d4-501b9164cfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:45.985715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4e0c342d-6767-4b71-81d4-501b9164cfa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:46.055091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:46.103938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c342d-6767-4b71-81d4-501b9164cfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:04:46.105428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c342d-6767-4b71-81d4-501b9164cfa7 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:04:46.125016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 668/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:04:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:04:49.049243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b97ba-96a9-4bbc-861b-9c3750296a8f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:49.049826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b97ba-96a9-4bbc-861b-9c3750296a8f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:49.049826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b97ba-96a9-4bbc-861b-9c3750296a8f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:49.051186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 652/1320] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:49.059178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed8fd-d46f-42cf-86a1-2917f05205d4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:49.059623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed8fd-d46f-42cf-86a1-2917f05205d4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:49.059822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed8fd-d46f-42cf-86a1-2917f05205d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:49.138947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cb7ed8fd-d46f-42cf-86a1-2917f05205d4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:49.140251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cb7ed8fd-d46f-42cf-86a1-2917f05205d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:49.243324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 669/1321] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:04:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:04:49.289643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b97ba-96a9-4bbc-861b-9c3750296a8f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:04:49.291201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-659b97ba-96a9-4bbc-861b-9c3750296a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:49.344998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:49.404099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b97ba-96a9-4bbc-861b-9c3750296a8f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:04:49.483055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b7045-b7ee-45f9-a398-6b56623dbd52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:04:49.484005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b7045-b7ee-45f9-a398-6b56623dbd52 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:49.484295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b7045-b7ee-45f9-a398-6b56623dbd52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:49.484564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b7045-b7ee-45f9-a398-6b56623dbd52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:49.486231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5b1b7045-b7ee-45f9-a398-6b56623dbd52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:49.600254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:49.614651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b7045-b7ee-45f9-a398-6b56623dbd52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:04:49.616103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b7045-b7ee-45f9-a398-6b56623dbd52 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:04:49.634729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 653/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:04:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:04:52.436386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6579fe-4ed9-46c7-9100-85ff710c95ff None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:52.436939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6579fe-4ed9-46c7-9100-85ff710c95ff None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:52.436939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6579fe-4ed9-46c7-9100-85ff710c95ff None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:52.438578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 670/1323] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:52.444183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306f35-466b-420b-89ca-0667a3a0f4d6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:52.444618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306f35-466b-420b-89ca-0667a3a0f4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:52.444712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306f35-466b-420b-89ca-0667a3a0f4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:52.516067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a0306f35-466b-420b-89ca-0667a3a0f4d6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:52.517341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a0306f35-466b-420b-89ca-0667a3a0f4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:52.624704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 654/1324] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:04:52.662265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6579fe-4ed9-46c7-9100-85ff710c95ff None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:04:52.663817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ce6579fe-4ed9-46c7-9100-85ff710c95ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:52.683789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:52.773586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6579fe-4ed9-46c7-9100-85ff710c95ff None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:04:52.849940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbd73e-64d7-44e6-84c5-bb8a5962e17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:04:52.850600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbd73e-64d7-44e6-84c5-bb8a5962e17c service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:52.850878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbd73e-64d7-44e6-84c5-bb8a5962e17c service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:52.851149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbd73e-64d7-44e6-84c5-bb8a5962e17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:52.852782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-11bbd73e-64d7-44e6-84c5-bb8a5962e17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:52.926984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:52.968276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbd73e-64d7-44e6-84c5-bb8a5962e17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:04:52.969764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbd73e-64d7-44e6-84c5-bb8a5962e17c service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:04:52.989851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 671/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:04:55.061849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af50096-fb63-43da-9a41-dc89e0ba1d46 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:55.062415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af50096-fb63-43da-9a41-dc89e0ba1d46 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:55.062415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af50096-fb63-43da-9a41-dc89e0ba1d46 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:55.063964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 655/1326] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:55.069447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f42e52-80c1-43ef-8b97-341082dc03fa None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:55.069897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f42e52-80c1-43ef-8b97-341082dc03fa None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:55.070022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f42e52-80c1-43ef-8b97-341082dc03fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:55.139269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e0f42e52-80c1-43ef-8b97-341082dc03fa None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:55.140565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e0f42e52-80c1-43ef-8b97-341082dc03fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:55.237248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 672/1327] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:04:55.272615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af50096-fb63-43da-9a41-dc89e0ba1d46 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:04:55.274203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9af50096-fb63-43da-9a41-dc89e0ba1d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:55.340126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:55.393854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af50096-fb63-43da-9a41-dc89e0ba1d46 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:04:55.472236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d2d5e-70ed-453b-ab82-5d3ea956cb52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:04:55.472896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2d5e-70ed-453b-ab82-5d3ea956cb52 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:55.473270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2d5e-70ed-453b-ab82-5d3ea956cb52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:55.473542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2d5e-70ed-453b-ab82-5d3ea956cb52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:55.475193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6d0d2d5e-70ed-453b-ab82-5d3ea956cb52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:55.590664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:55.593649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d2d5e-70ed-453b-ab82-5d3ea956cb52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:04:55.595466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d2d5e-70ed-453b-ab82-5d3ea956cb52 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:04:55.614620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 656/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:04:57.697526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80435d-0e3b-4ba5-823f-840573dde515 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:57.698045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80435d-0e3b-4ba5-823f-840573dde515 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:57.698045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80435d-0e3b-4ba5-823f-840573dde515 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:57.699435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 673/1329] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:04:57.705205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5022a0d-791f-408b-8894-f9e56890bd3c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:57.705677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5022a0d-791f-408b-8894-f9e56890bd3c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:57.705795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5022a0d-791f-408b-8894-f9e56890bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:57.782145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b5022a0d-791f-408b-8894-f9e56890bd3c None None] MFA Rules not processed for user `4b27ececfd83412c9c497073beaf72f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:04:57.783399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b5022a0d-791f-408b-8894-f9e56890bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:57.882444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 657/1330] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:04:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:04:57.926980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80435d-0e3b-4ba5-823f-840573dde515 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:04:57.928707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1a80435d-0e3b-4ba5-823f-840573dde515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:57.969993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:58.059881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80435d-0e3b-4ba5-823f-840573dde515 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:04:58.163288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad86b8-9af4-4716-9c65-198bdd12c04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:04:58.163959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad86b8-9af4-4716-9c65-198bdd12c04c service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:04:58.164290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad86b8-9af4-4716-9c65-198bdd12c04c service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:04:58.164510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad86b8-9af4-4716-9c65-198bdd12c04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:04:58.167052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6bad86b8-9af4-4716-9c65-198bdd12c04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:04:58.254682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:04:58.286518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad86b8-9af4-4716-9c65-198bdd12c04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:04:58.287976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad86b8-9af4-4716-9c65-198bdd12c04c service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:04:58.307736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 674/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:04:57 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:00.553952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cac09c-74e7-4c64-8a8b-a7dfd349eb93 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:00.554853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cac09c-74e7-4c64-8a8b-a7dfd349eb93 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:00.554853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cac09c-74e7-4c64-8a8b-a7dfd349eb93 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:00.557113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 658/1332] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:00.565004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb77bd-f6dd-433d-ae4d-2597fc409a2e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:00.565474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb77bd-f6dd-433d-ae4d-2597fc409a2e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:00.565570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb77bd-f6dd-433d-ae4d-2597fc409a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:00.632944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a4bb77bd-f6dd-433d-ae4d-2597fc409a2e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:00.634189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a4bb77bd-f6dd-433d-ae4d-2597fc409a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:00.738149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 675/1333] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:05:00.777566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cac09c-74e7-4c64-8a8b-a7dfd349eb93 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:00.779106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-53cac09c-74e7-4c64-8a8b-a7dfd349eb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:00.859114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:00.894229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cac09c-74e7-4c64-8a8b-a7dfd349eb93 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:00.977593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d052714-8297-4b5b-b4d7-7726b2f58ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:00.978243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d052714-8297-4b5b-b4d7-7726b2f58ddd service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:00.978527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d052714-8297-4b5b-b4d7-7726b2f58ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:00.978795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d052714-8297-4b5b-b4d7-7726b2f58ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:00.980417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2d052714-8297-4b5b-b4d7-7726b2f58ddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:01.096734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d052714-8297-4b5b-b4d7-7726b2f58ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:01.098291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d052714-8297-4b5b-b4d7-7726b2f58ddd service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:01.109225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:01.118180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 659/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:03.183700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573acd7-cfc0-4c6a-aee9-00356808e01d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:03.184155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573acd7-cfc0-4c6a-aee9-00356808e01d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:03.184251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573acd7-cfc0-4c6a-aee9-00356808e01d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:03.185655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 676/1335] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:03.192409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887754f4-123d-4f34-87fe-ad45c978bd73 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:03.192874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887754f4-123d-4f34-87fe-ad45c978bd73 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:03.192971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887754f4-123d-4f34-87fe-ad45c978bd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:03.267628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-887754f4-123d-4f34-87fe-ad45c978bd73 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:03.268754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-887754f4-123d-4f34-87fe-ad45c978bd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:03.378709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 660/1336] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:03.419317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573acd7-cfc0-4c6a-aee9-00356808e01d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:03.420955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a573acd7-cfc0-4c6a-aee9-00356808e01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:03.462673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:03.532002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573acd7-cfc0-4c6a-aee9-00356808e01d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:03.612115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01078f-2bb5-4248-9dbb-c07b3c499659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:03.612764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01078f-2bb5-4248-9dbb-c07b3c499659 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:03.613091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01078f-2bb5-4248-9dbb-c07b3c499659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:03.613352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01078f-2bb5-4248-9dbb-c07b3c499659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:03.614996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ac01078f-2bb5-4248-9dbb-c07b3c499659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:03.703208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:03.738997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01078f-2bb5-4248-9dbb-c07b3c499659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:03.740453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01078f-2bb5-4248-9dbb-c07b3c499659 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:03.760610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 677/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:06.269178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e73c3-561f-4c19-987b-15ef4ee47c57 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:06.269739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e73c3-561f-4c19-987b-15ef4ee47c57 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:06.269739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e73c3-561f-4c19-987b-15ef4ee47c57 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:06.271381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 661/1338] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:06.278178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb7f30-faf0-42bb-b6a3-2fb82c37b51c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:06.278637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb7f30-faf0-42bb-b6a3-2fb82c37b51c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:06.278784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb7f30-faf0-42bb-b6a3-2fb82c37b51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:06.349166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-05fb7f30-faf0-42bb-b6a3-2fb82c37b51c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:06.350398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-05fb7f30-faf0-42bb-b6a3-2fb82c37b51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:06.461097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 678/1339] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:06.496723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e73c3-561f-4c19-987b-15ef4ee47c57 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:06.498277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c89e73c3-561f-4c19-987b-15ef4ee47c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:06.580677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:06.609644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e73c3-561f-4c19-987b-15ef4ee47c57 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:06.684428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f9be8-3355-4f00-b522-fb761125d855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:06.685089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9be8-3355-4f00-b522-fb761125d855 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:06.685371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9be8-3355-4f00-b522-fb761125d855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:06.685618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9be8-3355-4f00-b522-fb761125d855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:06.687429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8f3f9be8-3355-4f00-b522-fb761125d855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:06.805310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f9be8-3355-4f00-b522-fb761125d855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:06.806830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f9be8-3355-4f00-b522-fb761125d855 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:06.828828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 662/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:10.962339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a29b5-992f-45c1-a0cf-d3b65963826a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:10.962894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a29b5-992f-45c1-a0cf-d3b65963826a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:10.962993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a29b5-992f-45c1-a0cf-d3b65963826a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:10.964352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 679/1341] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:10.969887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f6d5-0b4a-4141-aabd-04fb8e0741ae None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:10.970317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f6d5-0b4a-4141-aabd-04fb8e0741ae None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:10.970429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f6d5-0b4a-4141-aabd-04fb8e0741ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:10.974875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:11.039557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0a05f6d5-0b4a-4141-aabd-04fb8e0741ae None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:11.040827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0a05f6d5-0b4a-4141-aabd-04fb8e0741ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:11.140981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 663/1342] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:05:11.179703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a29b5-992f-45c1-a0cf-d3b65963826a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:11.181411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-953a29b5-992f-45c1-a0cf-d3b65963826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:11.230999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:11.288997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a29b5-992f-45c1-a0cf-d3b65963826a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:11.370987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9dc9b-34a6-4b8f-91b3-770b0218684a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:11.371554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9dc9b-34a6-4b8f-91b3-770b0218684a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:11.371832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9dc9b-34a6-4b8f-91b3-770b0218684a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:11.372117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9dc9b-34a6-4b8f-91b3-770b0218684a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:11.373840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8b9dc9b-34a6-4b8f-91b3-770b0218684a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:11.466572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:11.484432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9dc9b-34a6-4b8f-91b3-770b0218684a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:11.485902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9dc9b-34a6-4b8f-91b3-770b0218684a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:11.504909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 680/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:14.122484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962925a6-9ed5-456e-af0b-c8dbfd460ae5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:14.123173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962925a6-9ed5-456e-af0b-c8dbfd460ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:14.123319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962925a6-9ed5-456e-af0b-c8dbfd460ae5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:14.124701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 664/1344] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:14.130068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ce09-7768-4a82-b6e6-8b5c7a802be1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:14.130541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ce09-7768-4a82-b6e6-8b5c7a802be1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:14.130654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ce09-7768-4a82-b6e6-8b5c7a802be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:14.202602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4270ce09-7768-4a82-b6e6-8b5c7a802be1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:14.203831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4270ce09-7768-4a82-b6e6-8b5c7a802be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:14.315526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 681/1345] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:14.355779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962925a6-9ed5-456e-af0b-c8dbfd460ae5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:14.357477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-962925a6-9ed5-456e-af0b-c8dbfd460ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:14.452173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:14.470075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962925a6-9ed5-456e-af0b-c8dbfd460ae5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:14.544784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830f1b0-8202-44a5-96d5-f07518e380d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:14.545636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f1b0-8202-44a5-96d5-f07518e380d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:14.545924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f1b0-8202-44a5-96d5-f07518e380d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:14.546198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830f1b0-8202-44a5-96d5-f07518e380d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:14.547843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9830f1b0-8202-44a5-96d5-f07518e380d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:14.667362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830f1b0-8202-44a5-96d5-f07518e380d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:14.668698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830f1b0-8202-44a5-96d5-f07518e380d7 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:14.687891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 665/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:18.830593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58b54-1efe-4e85-9d15-114d7bc8768e None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:18.831159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58b54-1efe-4e85-9d15-114d7bc8768e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:18.831159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58b54-1efe-4e85-9d15-114d7bc8768e None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:18.832582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 682/1347] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:18.838114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27e1b-df77-4046-afd0-da5842337c09 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:18.838559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27e1b-df77-4046-afd0-da5842337c09 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:18.838667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27e1b-df77-4046-afd0-da5842337c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:18.853677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:18.906579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-39f27e1b-df77-4046-afd0-da5842337c09 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:18.907807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39f27e1b-df77-4046-afd0-da5842337c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:19.012565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 666/1348] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:19.051298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58b54-1efe-4e85-9d15-114d7bc8768e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:19.053672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-77b58b54-1efe-4e85-9d15-114d7bc8768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:19.134413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:19.189559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58b54-1efe-4e85-9d15-114d7bc8768e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:19.282027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9f6b0-ecb9-4742-8256-b5c4e60323d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:19.282704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f6b0-ecb9-4742-8256-b5c4e60323d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:19.283822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f6b0-ecb9-4742-8256-b5c4e60323d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:19.284108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f6b0-ecb9-4742-8256-b5c4e60323d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:19.285989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8cd9f6b0-ecb9-4742-8256-b5c4e60323d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:19.412851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:19.417968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9f6b0-ecb9-4742-8256-b5c4e60323d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:19.420224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9f6b0-ecb9-4742-8256-b5c4e60323d1 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:19.444626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 683/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:24.463436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c0be1-ffc3-4810-b621-5a648f8f9e9d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:24.463961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c0be1-ffc3-4810-b621-5a648f8f9e9d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:24.464045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c0be1-ffc3-4810-b621-5a648f8f9e9d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:24.465851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 667/1350] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:05:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:24.485003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3e87-a1c2-492c-b48d-9c5711192051 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:24.485520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3e87-a1c2-492c-b48d-9c5711192051 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:24.485751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3e87-a1c2-492c-b48d-9c5711192051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:24.572971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1b2b3e87-a1c2-492c-b48d-9c5711192051 None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:24.574115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1b2b3e87-a1c2-492c-b48d-9c5711192051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:24.685709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 684/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:05:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:28.796363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271649fe-ead5-4ffe-a147-06310395d016 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:28.797305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271649fe-ead5-4ffe-a147-06310395d016 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:28.797305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271649fe-ead5-4ffe-a147-06310395d016 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:28.798592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 668/1352] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:28.804091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de579-303d-45e4-844d-0e0a506e87b0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:28.804992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de579-303d-45e4-844d-0e0a506e87b0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:28.805532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de579-303d-45e4-844d-0e0a506e87b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:28.864905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:28.891745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-132de579-303d-45e4-844d-0e0a506e87b0 None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:28.893089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-132de579-303d-45e4-844d-0e0a506e87b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:29.010688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=59 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:29.024271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 685/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:05:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:30.293430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd4a7-d18c-4481-b6e4-0cd5aac48894 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:30.293997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd4a7-d18c-4481-b6e4-0cd5aac48894 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:30.293997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd4a7-d18c-4481-b6e4-0cd5aac48894 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:30.295437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 669/1354] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:30.300530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2f5d-7c7c-4223-bb4a-cc3e785578d7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:30.300974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2f5d-7c7c-4223-bb4a-cc3e785578d7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:30.301253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2f5d-7c7c-4223-bb4a-cc3e785578d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:30.385511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7d0f2f5d-7c7c-4223-bb4a-cc3e785578d7 None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:30.386695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7d0f2f5d-7c7c-4223-bb4a-cc3e785578d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:30.429459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:30.495838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 686/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:05:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:05:34.004660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836d51b-6853-4b1d-b18b-bcfb51f54ec6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:34.005555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836d51b-6853-4b1d-b18b-bcfb51f54ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:34.005976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836d51b-6853-4b1d-b18b-bcfb51f54ec6 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:34.007533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 670/1356] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:34.013903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116e8c2-f950-472f-8eb3-f269428376b8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:34.014333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116e8c2-f950-472f-8eb3-f269428376b8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:34.014940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116e8c2-f950-472f-8eb3-f269428376b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:34.094048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2116e8c2-f950-472f-8eb3-f269428376b8 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:34.095316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2116e8c2-f950-472f-8eb3-f269428376b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:34.201118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 687/1357] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:34.254068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64f6d2-9aa3-41bb-8836-895ab42cd260 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:34.254329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64f6d2-9aa3-41bb-8836-895ab42cd260 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:34.254762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64f6d2-9aa3-41bb-8836-895ab42cd260 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:34.256297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 671/1358] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 07:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:34.269525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf72bb-85b9-48c8-bf69-e49bf43f00c9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:34.270184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf72bb-85b9-48c8-bf69-e49bf43f00c9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:34.271084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf72bb-85b9-48c8-bf69-e49bf43f00c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:34.296014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:34.362670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-31cf72bb-85b9-48c8-bf69-e49bf43f00c9 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:34.364220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-31cf72bb-85b9-48c8-bf69-e49bf43f00c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:34.491849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 688/1359] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 07:05:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:05:34.500865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64f6d2-9aa3-41bb-8836-895ab42cd260 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:34.502701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f64f6d2-9aa3-41bb-8836-895ab42cd260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:34.657812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64f6d2-9aa3-41bb-8836-895ab42cd260 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:34.732767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:34.780316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257c558-2b6f-40fd-9f0c-27e08fa04e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:34.781248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257c558-2b6f-40fd-9f0c-27e08fa04e99 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:34.781609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257c558-2b6f-40fd-9f0c-27e08fa04e99 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:34.781946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257c558-2b6f-40fd-9f0c-27e08fa04e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:34.783658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5257c558-2b6f-40fd-9f0c-27e08fa04e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:34.877152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257c558-2b6f-40fd-9f0c-27e08fa04e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:34.878853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257c558-2b6f-40fd-9f0c-27e08fa04e99 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:34.898106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 672/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:37.177830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50834a4a-6782-4b15-bb70-9938428b5905 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:37.178451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50834a4a-6782-4b15-bb70-9938428b5905 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:37.178451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50834a4a-6782-4b15-bb70-9938428b5905 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:37.179996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 689/1361] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:37.185621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb61ce3-fcd9-47bb-9fef-7e5c5abbcdf2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:37.186221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb61ce3-fcd9-47bb-9fef-7e5c5abbcdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:37.186221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb61ce3-fcd9-47bb-9fef-7e5c5abbcdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:37.256428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0eb61ce3-fcd9-47bb-9fef-7e5c5abbcdf2 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:37.257754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0eb61ce3-fcd9-47bb-9fef-7e5c5abbcdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:37.299187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:37.370599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 673/1362] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:37.410352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50834a4a-6782-4b15-bb70-9938428b5905 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:37.412072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-50834a4a-6782-4b15-bb70-9938428b5905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:37.500215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:37.594189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50834a4a-6782-4b15-bb70-9938428b5905 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:37.740610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89845ab-620a-4f90-baaf-b8a8cf5ee59c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:37.741433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89845ab-620a-4f90-baaf-b8a8cf5ee59c service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:37.741721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89845ab-620a-4f90-baaf-b8a8cf5ee59c service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:37.741991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89845ab-620a-4f90-baaf-b8a8cf5ee59c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:37.743642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c89845ab-620a-4f90-baaf-b8a8cf5ee59c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:37.769564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:37.858889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89845ab-620a-4f90-baaf-b8a8cf5ee59c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:37.860729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89845ab-620a-4f90-baaf-b8a8cf5ee59c service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:37.881318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 690/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:40.246420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76997c0-09dc-4959-a5b5-064db94cb43f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:40.247069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76997c0-09dc-4959-a5b5-064db94cb43f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:40.247181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76997c0-09dc-4959-a5b5-064db94cb43f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:40.248687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 674/1364] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:40.254191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d617bd-41c1-4c03-96aa-a5afb4fe0536 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:40.255021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d617bd-41c1-4c03-96aa-a5afb4fe0536 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:40.255538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d617bd-41c1-4c03-96aa-a5afb4fe0536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:40.325880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c7d617bd-41c1-4c03-96aa-a5afb4fe0536 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:40.327671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c7d617bd-41c1-4c03-96aa-a5afb4fe0536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:40.428422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:40.431304 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 691/1365] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:40.476429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d96495-3b33-47c4-957f-04e3d5104c2f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:40.476718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d96495-3b33-47c4-957f-04e3d5104c2f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:40.476990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d96495-3b33-47c4-957f-04e3d5104c2f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:40.478529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 675/1366] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 07:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:40.484464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb8162-2ee0-4959-843f-16e60c52f967 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:40.484754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb8162-2ee0-4959-843f-16e60c52f967 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:40.485030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb8162-2ee0-4959-843f-16e60c52f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:40.565032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cdeb8162-2ee0-4959-843f-16e60c52f967 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:40.566661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cdeb8162-2ee0-4959-843f-16e60c52f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:40.696581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 692/1367] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 07:05:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:40.704816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d96495-3b33-47c4-957f-04e3d5104c2f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:40.706414 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39d96495-3b33-47c4-957f-04e3d5104c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:40.864636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d96495-3b33-47c4-957f-04e3d5104c2f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:40.885656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:40.990520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82007471-1e10-4af3-8e18-6f51cf8d0374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:40.990520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82007471-1e10-4af3-8e18-6f51cf8d0374 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:40.990520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82007471-1e10-4af3-8e18-6f51cf8d0374 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:40.990520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82007471-1e10-4af3-8e18-6f51cf8d0374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:40.990993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-82007471-1e10-4af3-8e18-6f51cf8d0374 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:41.089865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82007471-1e10-4af3-8e18-6f51cf8d0374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:41.091666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82007471-1e10-4af3-8e18-6f51cf8d0374 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:41.112154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 676/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:42.068588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d6d8-02ea-4a98-9983-f2bf67ae3f14 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:42.069162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d6d8-02ea-4a98-9983-f2bf67ae3f14 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:42.069162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d6d8-02ea-4a98-9983-f2bf67ae3f14 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:42.070631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 693/1369] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:42.076090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00fed4-3f55-45c0-bf46-3c26c38a24b3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:42.076555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00fed4-3f55-45c0-bf46-3c26c38a24b3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:42.076665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00fed4-3f55-45c0-bf46-3c26c38a24b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:42.110063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:42.148160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4b00fed4-3f55-45c0-bf46-3c26c38a24b3 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:42.149410 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4b00fed4-3f55-45c0-bf46-3c26c38a24b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:42.257227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 677/1370] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:42.604080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8d15-6f04-4c44-b715-c6fff720afff None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:42.604080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8d15-6f04-4c44-b715-c6fff720afff None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:42.604080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8d15-6f04-4c44-b715-c6fff720afff None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:42.609082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 694/1371] 10.222.0.25 () {54 vars in 963 bytes} [Tue Jul 12 07:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:42.610852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefec98-2b57-433f-9d73-5b6281661fe9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:42.611098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefec98-2b57-433f-9d73-5b6281661fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:42.611400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefec98-2b57-433f-9d73-5b6281661fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:42.691347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7aefec98-2b57-433f-9d73-5b6281661fe9 None None] MFA Rules not processed for user `b4292042e79c45438b5964710359cd5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:42.692585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7aefec98-2b57-433f-9d73-5b6281661fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:42.743472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:42.778133 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 678/1372] 10.222.0.25 () {58 vars in 1060 bytes} [Tue Jul 12 07:05:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:05:42.785826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc8d15-6f04-4c44-b715-c6fff720afff None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:42.787456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-efcc8d15-6f04-4c44-b715-c6fff720afff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:42.847309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:42.911684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc8d15-6f04-4c44-b715-c6fff720afff None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:42.989875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44053c6-3b7d-49d1-ba28-0e73ca0aeb63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:42.990729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44053c6-3b7d-49d1-ba28-0e73ca0aeb63 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:42.991109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44053c6-3b7d-49d1-ba28-0e73ca0aeb63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:42.991500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44053c6-3b7d-49d1-ba28-0e73ca0aeb63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:43.001306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f44053c6-3b7d-49d1-ba28-0e73ca0aeb63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:43.120360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:43.137206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44053c6-3b7d-49d1-ba28-0e73ca0aeb63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:43.138925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44053c6-3b7d-49d1-ba28-0e73ca0aeb63 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:43.159388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 695/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:43.268413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabdcc-64ee-4fcf-9f33-70e3d1fdee3b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:43.268969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabdcc-64ee-4fcf-9f33-70e3d1fdee3b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:43.268969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabdcc-64ee-4fcf-9f33-70e3d1fdee3b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:43.273554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 679/1374] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:43.275679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7c410-53e2-4f38-850d-c8c4db01a3fe None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:43.275886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7c410-53e2-4f38-850d-c8c4db01a3fe None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:43.276439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7c410-53e2-4f38-850d-c8c4db01a3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:43.342642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2cb7c410-53e2-4f38-850d-c8c4db01a3fe None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:43.343678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2cb7c410-53e2-4f38-850d-c8c4db01a3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:43.466402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 696/1375] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:43.510516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cabdcc-64ee-4fcf-9f33-70e3d1fdee3b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:43.512053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-36cabdcc-64ee-4fcf-9f33-70e3d1fdee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:43.659609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cabdcc-64ee-4fcf-9f33-70e3d1fdee3b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:43.712729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:43.774581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20ac60-2b25-48f2-9f39-184915860c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:43.775274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20ac60-2b25-48f2-9f39-184915860c4f service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:43.775552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20ac60-2b25-48f2-9f39-184915860c4f service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:43.775820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20ac60-2b25-48f2-9f39-184915860c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:43.777604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bb20ac60-2b25-48f2-9f39-184915860c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:43.873494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20ac60-2b25-48f2-9f39-184915860c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:43.875078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20ac60-2b25-48f2-9f39-184915860c4f service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:43.894145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 680/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:45.936715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14ec4-b4ab-4d28-9a44-6ae403bb6706 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:45.937310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14ec4-b4ab-4d28-9a44-6ae403bb6706 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:45.937310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14ec4-b4ab-4d28-9a44-6ae403bb6706 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:45.938724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 697/1377] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:45.944500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa5889-9217-4d52-9fad-8b0c92248599 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:45.944965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa5889-9217-4d52-9fad-8b0c92248599 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:45.945273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa5889-9217-4d52-9fad-8b0c92248599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:46.013959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e5aa5889-9217-4d52-9fad-8b0c92248599 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:46.015172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5aa5889-9217-4d52-9fad-8b0c92248599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:46.024589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:46.162973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 681/1378] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:05:46.206971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14ec4-b4ab-4d28-9a44-6ae403bb6706 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:46.208675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-21e14ec4-b4ab-4d28-9a44-6ae403bb6706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:46.276972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:46.372507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14ec4-b4ab-4d28-9a44-6ae403bb6706 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:46.511650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d4242-2e17-4014-b18a-e0a56ff70f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:46.512475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d4242-2e17-4014-b18a-e0a56ff70f2f service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:46.513358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d4242-2e17-4014-b18a-e0a56ff70f2f service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:46.513708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d4242-2e17-4014-b18a-e0a56ff70f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:46.515429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-389d4242-2e17-4014-b18a-e0a56ff70f2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:46.538524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:46.642142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d4242-2e17-4014-b18a-e0a56ff70f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:46.643705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d4242-2e17-4014-b18a-e0a56ff70f2f service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:46.674318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 698/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:48.739249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ee59-b62b-4f6d-b96f-4c94b73b497e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:48.739824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ee59-b62b-4f6d-b96f-4c94b73b497e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:48.739824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ee59-b62b-4f6d-b96f-4c94b73b497e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:48.741353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 682/1380] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:48.746339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c313-7344-4efd-9893-239c40a0f0a9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:48.746955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c313-7344-4efd-9893-239c40a0f0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:48.747075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c313-7344-4efd-9893-239c40a0f0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:48.816256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-43f3c313-7344-4efd-9893-239c40a0f0a9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:48.817458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-43f3c313-7344-4efd-9893-239c40a0f0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:48.917356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:48.924613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 699/1381] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:48.968035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8ee59-b62b-4f6d-b96f-4c94b73b497e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:48.969959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eeb8ee59-b62b-4f6d-b96f-4c94b73b497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:49.113922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:49.126309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8ee59-b62b-4f6d-b96f-4c94b73b497e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:49.244489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cd35c-4420-42a8-9561-d0437ecc9d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:49.245258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cd35c-4420-42a8-9561-d0437ecc9d43 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:49.245637 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cd35c-4420-42a8-9561-d0437ecc9d43 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:49.246030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cd35c-4420-42a8-9561-d0437ecc9d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:49.248223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d49cd35c-4420-42a8-9561-d0437ecc9d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:49.343465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cd35c-4420-42a8-9561-d0437ecc9d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:49.345238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cd35c-4420-42a8-9561-d0437ecc9d43 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:49.356453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:49.366139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 683/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:51.559024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dd990-5444-4cfb-ae26-5421a9458d27 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:51.559708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dd990-5444-4cfb-ae26-5421a9458d27 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:51.559708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dd990-5444-4cfb-ae26-5421a9458d27 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:51.561329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 700/1383] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:51.568127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e1443-4900-4f28-8c81-d2bea98a1f2f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:51.568626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e1443-4900-4f28-8c81-d2bea98a1f2f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:51.568827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e1443-4900-4f28-8c81-d2bea98a1f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:51.657969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9f7e1443-4900-4f28-8c81-d2bea98a1f2f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:51.659273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9f7e1443-4900-4f28-8c81-d2bea98a1f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:51.773363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 684/1384] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:51.820473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dd990-5444-4cfb-ae26-5421a9458d27 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:51.823021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-846dd990-5444-4cfb-ae26-5421a9458d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:51.988322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dd990-5444-4cfb-ae26-5421a9458d27 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:52.070886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:52.128845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfdd52-448d-4d17-b70b-dabe3ba479b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:52.129685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfdd52-448d-4d17-b70b-dabe3ba479b4 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:52.129970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfdd52-448d-4d17-b70b-dabe3ba479b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:52.130168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfdd52-448d-4d17-b70b-dabe3ba479b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:52.132966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98cfdd52-448d-4d17-b70b-dabe3ba479b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:52.247237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfdd52-448d-4d17-b70b-dabe3ba479b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:52.248823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfdd52-448d-4d17-b70b-dabe3ba479b4 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:52.270872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 701/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:54.926504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd32da-7aad-4bb3-8091-bc0cdadc1c2b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:54.927162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd32da-7aad-4bb3-8091-bc0cdadc1c2b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:54.927162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd32da-7aad-4bb3-8091-bc0cdadc1c2b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:54.928847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 685/1386] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:54.936046 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450697d-12ce-4d05-bec4-fe1211e9565f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:54.936501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450697d-12ce-4d05-bec4-fe1211e9565f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:54.936895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450697d-12ce-4d05-bec4-fe1211e9565f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:55.018370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7450697d-12ce-4d05-bec4-fe1211e9565f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:55.020250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7450697d-12ce-4d05-bec4-fe1211e9565f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:55.063181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:55.132419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 702/1387] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:55.181102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd32da-7aad-4bb3-8091-bc0cdadc1c2b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:55.183347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5dbd32da-7aad-4bb3-8091-bc0cdadc1c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:55.287267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:55.373230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd32da-7aad-4bb3-8091-bc0cdadc1c2b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:55.515972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb929ae-3c94-409e-a899-e3632a38827e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:55.517202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb929ae-3c94-409e-a899-e3632a38827e service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:55.517561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb929ae-3c94-409e-a899-e3632a38827e service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:55.517912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb929ae-3c94-409e-a899-e3632a38827e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:55.520142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eeb929ae-3c94-409e-a899-e3632a38827e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:55.570867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:55.633136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb929ae-3c94-409e-a899-e3632a38827e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:55.634793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb929ae-3c94-409e-a899-e3632a38827e service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:55.656146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 686/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:57.714200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a2bd0-bdd9-4c67-9d72-16461edfbd72 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:57.714806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a2bd0-bdd9-4c67-9d72-16461edfbd72 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:57.714806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a2bd0-bdd9-4c67-9d72-16461edfbd72 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:57.716530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 703/1389] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:57.721995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfac08-61ae-41c6-9787-eebe330886c0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:57.722524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfac08-61ae-41c6-9787-eebe330886c0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:57.722524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfac08-61ae-41c6-9787-eebe330886c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:57.790020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-eebfac08-61ae-41c6-9787-eebe330886c0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:57.791201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eebfac08-61ae-41c6-9787-eebe330886c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:57.892429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 687/1390] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:57.936437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a2bd0-bdd9-4c67-9d72-16461edfbd72 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:57.938018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ea9a2bd0-bdd9-4c67-9d72-16461edfbd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:58.092637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a2bd0-bdd9-4c67-9d72-16461edfbd72 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:58.109422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:58.173237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b0e7c-b4a0-49ed-b494-76acdfa5da79 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:58.173513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b0e7c-b4a0-49ed-b494-76acdfa5da79 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:58.173788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b0e7c-b4a0-49ed-b494-76acdfa5da79 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:58.175205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 688/1391] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:05:58.180325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2417-57a9-4d84-bc57-26c4e0a86db1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:58.180610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2417-57a9-4d84-bc57-26c4e0a86db1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:58.180887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2417-57a9-4d84-bc57-26c4e0a86db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:58.209629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:58.212812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071434-9793-48a7-a676-bd1a4e4a810b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:58.213714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071434-9793-48a7-a676-bd1a4e4a810b service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:58.214152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071434-9793-48a7-a676-bd1a4e4a810b service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:58.214574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071434-9793-48a7-a676-bd1a4e4a810b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:58.217676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-88071434-9793-48a7-a676-bd1a4e4a810b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:58.268776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-569a2417-57a9-4d84-bc57-26c4e0a86db1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:58.269932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-569a2417-57a9-4d84-bc57-26c4e0a86db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:58.319935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071434-9793-48a7-a676-bd1a4e4a810b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:58.321778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071434-9793-48a7-a676-bd1a4e4a810b service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:58.341237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 704/1392] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:58.377148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 689/1393] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:58.440671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa48a2-047b-48d3-b3fd-5c645682ca4f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:58.440923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa48a2-047b-48d3-b3fd-5c645682ca4f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:58.441329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa48a2-047b-48d3-b3fd-5c645682ca4f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:58.443124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 705/1394] 10.222.0.25 () {54 vars in 962 bytes} [Tue Jul 12 07:05:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:58.455795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fecf13-bcac-4a79-90ca-f2a60c11081a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:58.456134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fecf13-bcac-4a79-90ca-f2a60c11081a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:58.456357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fecf13-bcac-4a79-90ca-f2a60c11081a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:58.549298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-27fecf13-bcac-4a79-90ca-f2a60c11081a None None] MFA Rules not processed for user `366a7e7596d14b32a6ce62790a2e3dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:58.550505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-27fecf13-bcac-4a79-90ca-f2a60c11081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:58.589617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:58.632621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 690/1395] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Jul 12 07:05:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:58.640690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa48a2-047b-48d3-b3fd-5c645682ca4f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:05:58.642306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4ffa48a2-047b-48d3-b3fd-5c645682ca4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:58.667805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:58.759291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa48a2-047b-48d3-b3fd-5c645682ca4f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:05:58.836951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bd4c0-ff8b-4978-8df9-69a50ed4b821 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366a7e7596d14b32a6ce62790a2e3dd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:05:58.837543 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd4c0-ff8b-4978-8df9-69a50ed4b821 service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:58.837819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd4c0-ff8b-4978-8df9-69a50ed4b821 service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:58.838082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd4c0-ff8b-4978-8df9-69a50ed4b821 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:58.839708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7c1bd4c0-ff8b-4978-8df9-69a50ed4b821 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:58.917230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:05:58.956324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bd4c0-ff8b-4978-8df9-69a50ed4b821 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:05:58.957803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bd4c0-ff8b-4978-8df9-69a50ed4b821 service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:05:58.976611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 706/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:05:59.716917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece52bc-aba2-4eb9-8f10-09931929eb5b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:59.717506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece52bc-aba2-4eb9-8f10-09931929eb5b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:59.717506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece52bc-aba2-4eb9-8f10-09931929eb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:59.786701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fece52bc-aba2-4eb9-8f10-09931929eb5b None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:05:59.787969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fece52bc-aba2-4eb9-8f10-09931929eb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:05:59.878360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 691/1397] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:05:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:05:59.920256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370cb8d-dadd-4fc0-9998-908d00c5d16a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:59.920812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370cb8d-dadd-4fc0-9998-908d00c5d16a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:59.920812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370cb8d-dadd-4fc0-9998-908d00c5d16a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:05:59.922258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 707/1398] 10.222.0.25 () {54 vars in 942 bytes} [Tue Jul 12 07:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:05:59.928696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fba01-dd9d-4ac1-abe0-c140050139c3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:05:59.928981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fba01-dd9d-4ac1-abe0-c140050139c3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:05:59.929272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fba01-dd9d-4ac1-abe0-c140050139c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:00.110193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:00.129535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ec4fba01-dd9d-4ac1-abe0-c140050139c3 None None] MFA Rules not processed for user `de86f0a9c2d14cc0b26b19ed719529ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:00.130745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ec4fba01-dd9d-4ac1-abe0-c140050139c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:00.244379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 692/1399] 10.222.0.25 () {58 vars in 1039 bytes} [Tue Jul 12 07:05:59 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:00.254480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370cb8d-dadd-4fc0-9998-908d00c5d16a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:00.256193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c370cb8d-dadd-4fc0-9998-908d00c5d16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:00.399362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370cb8d-dadd-4fc0-9998-908d00c5d16a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:00.431793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee84a4-0d53-464a-af6d-a7ef7eaa0cba None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:00.431995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee84a4-0d53-464a-af6d-a7ef7eaa0cba None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:00.432218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee84a4-0d53-464a-af6d-a7ef7eaa0cba None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:00.433868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 693/1400] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:06:00.438778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:00.439052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:00.439682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:00.444060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:00.503402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:00.504606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:00.509376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24f592-d75d-4944-a461-46430c6b9c8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:00.510077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f592-d75d-4944-a461-46430c6b9c8c service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:00.510478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f592-d75d-4944-a461-46430c6b9c8c service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:00.510761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f592-d75d-4944-a461-46430c6b9c8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:00.512537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3e24f592-d75d-4944-a461-46430c6b9c8c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:00.561823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:00.597535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24f592-d75d-4944-a461-46430c6b9c8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:00.599171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24f592-d75d-4944-a461-46430c6b9c8c service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:00.605322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 694/1401] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:06:00.607602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 708/1402] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:06:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:06:00.650908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:00.652134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:00.791522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da7040-8cf1-428e-a91c-4c1a22786efe None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:00.868961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:00.920362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69f6f9-57be-4d77-9c92-fa9593edc804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:00.921018 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69f6f9-57be-4d77-9c92-fa9593edc804 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:00.921324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69f6f9-57be-4d77-9c92-fa9593edc804 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:00.921593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69f6f9-57be-4d77-9c92-fa9593edc804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:00.923255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7c69f6f9-57be-4d77-9c92-fa9593edc804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:01.032507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69f6f9-57be-4d77-9c92-fa9593edc804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:01.034187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69f6f9-57be-4d77-9c92-fa9593edc804 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:01.059211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 695/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:03.129234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bf6c-6ecf-4b34-8a3f-f562dd8cc438 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:03.130351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bf6c-6ecf-4b34-8a3f-f562dd8cc438 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:03.130708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bf6c-6ecf-4b34-8a3f-f562dd8cc438 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:03.132203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 709/1404] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:03.137503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc45bd-5080-4eb4-8f11-38dbf4d0962a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:03.137948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc45bd-5080-4eb4-8f11-38dbf4d0962a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:03.138044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc45bd-5080-4eb4-8f11-38dbf4d0962a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:03.210278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-37cc45bd-5080-4eb4-8f11-38dbf4d0962a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:03.211554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-37cc45bd-5080-4eb4-8f11-38dbf4d0962a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:03.246236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:03.316579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 696/1405] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:03.362377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bf6c-6ecf-4b34-8a3f-f562dd8cc438 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:03.364185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9224bf6c-6ecf-4b34-8a3f-f562dd8cc438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:03.456139 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:03.534919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bf6c-6ecf-4b34-8a3f-f562dd8cc438 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:03.655442 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb5b2e-0cf2-4f30-bb70-161bc4491413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:03.656120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb5b2e-0cf2-4f30-bb70-161bc4491413 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:03.656407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb5b2e-0cf2-4f30-bb70-161bc4491413 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:03.656685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb5b2e-0cf2-4f30-bb70-161bc4491413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:03.658387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-64eb5b2e-0cf2-4f30-bb70-161bc4491413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:03.702699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:03.758435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb5b2e-0cf2-4f30-bb70-161bc4491413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:03.760352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb5b2e-0cf2-4f30-bb70-161bc4491413 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:03.789248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 710/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:05.865173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13af47-46c7-4b05-9f6f-0e9441f1a545 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:05.865173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13af47-46c7-4b05-9f6f-0e9441f1a545 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:05.865173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13af47-46c7-4b05-9f6f-0e9441f1a545 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:05.865173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 697/1407] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:05.870882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d5bfb-7271-4b66-807c-93b2597a2e2c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:05.871731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d5bfb-7271-4b66-807c-93b2597a2e2c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:05.872257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d5bfb-7271-4b66-807c-93b2597a2e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:05.964991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e13d5bfb-7271-4b66-807c-93b2597a2e2c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:05.970695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e13d5bfb-7271-4b66-807c-93b2597a2e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:06.093397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 711/1408] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:06.136595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13af47-46c7-4b05-9f6f-0e9441f1a545 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:06.140551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fd13af47-46c7-4b05-9f6f-0e9441f1a545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:06.312828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13af47-46c7-4b05-9f6f-0e9441f1a545 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:06.331509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:06.452866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b370031-a783-4568-9ff6-e13b15561a78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:06.453564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b370031-a783-4568-9ff6-e13b15561a78 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:06.453857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b370031-a783-4568-9ff6-e13b15561a78 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:06.454126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b370031-a783-4568-9ff6-e13b15561a78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:06.455773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6b370031-a783-4568-9ff6-e13b15561a78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:06.555520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b370031-a783-4568-9ff6-e13b15561a78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:06.556479 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b370031-a783-4568-9ff6-e13b15561a78 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:06.579110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 698/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:16.091323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:16.576313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:22.703579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49a220-927e-4ed4-8804-368e5a8364c2 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:22.704189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49a220-927e-4ed4-8804-368e5a8364c2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:22.704189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49a220-927e-4ed4-8804-368e5a8364c2 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:22.705587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 712/1410] 10.222.0.25 () {54 vars in 918 bytes} [Tue Jul 12 07:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:22.711086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70168b-2174-4948-a76f-80a686f717a8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:22.711520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70168b-2174-4948-a76f-80a686f717a8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:22.711626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70168b-2174-4948-a76f-80a686f717a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:22.788099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6c70168b-2174-4948-a76f-80a686f717a8 None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:22.789331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6c70168b-2174-4948-a76f-80a686f717a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:22.905902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 699/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Jul 12 07:06:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:22.924117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1140-40c2-4e8d-a9da-39b4eed0c8b3 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:22.924405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1140-40c2-4e8d-a9da-39b4eed0c8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:22.924706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1140-40c2-4e8d-a9da-39b4eed0c8b3 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:22.926137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 713/1412] 10.222.0.25 () {54 vars in 918 bytes} [Tue Jul 12 07:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:22.931129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17956b1-9893-479e-9d62-3681a4f68462 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:22.931412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17956b1-9893-479e-9d62-3681a4f68462 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:22.931685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17956b1-9893-479e-9d62-3681a4f68462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:23.012378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:23.015228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b17956b1-9893-479e-9d62-3681a4f68462 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:23.016778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b17956b1-9893-479e-9d62-3681a4f68462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:23.140602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 700/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Jul 12 07:06:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:06:25.712085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d544-5266-4632-88cd-5c5ed9aabef5 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:25.712675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d544-5266-4632-88cd-5c5ed9aabef5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:25.712675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d544-5266-4632-88cd-5c5ed9aabef5 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:25.714072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:25.720712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19984-f0bb-491d-babb-f6bf7ddb5f49 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:25.721439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19984-f0bb-491d-babb-f6bf7ddb5f49 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:25.721587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19984-f0bb-491d-babb-f6bf7ddb5f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:25.807023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-87d19984-f0bb-491d-babb-f6bf7ddb5f49 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:25.808422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-87d19984-f0bb-491d-babb-f6bf7ddb5f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:25.871954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:25.914669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:25.958561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202d544-5266-4632-88cd-5c5ed9aabef5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:25.960942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2202d544-5266-4632-88cd-5c5ed9aabef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:26.114939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202d544-5266-4632-88cd-5c5ed9aabef5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:26.201254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:26.245087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfffec5-8508-4652-ba01-ae9e202706e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:26.245914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfffec5-8508-4652-ba01-ae9e202706e0 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:26.246293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfffec5-8508-4652-ba01-ae9e202706e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:26.246576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfffec5-8508-4652-ba01-ae9e202706e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:26.248238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0dfffec5-8508-4652-ba01-ae9e202706e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:26.341289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfffec5-8508-4652-ba01-ae9e202706e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:26.342876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfffec5-8508-4652-ba01-ae9e202706e0 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:26.361281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 715/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:27.258386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ade657-50a5-4481-9b13-c28e1bb55be1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:27.259150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ade657-50a5-4481-9b13-c28e1bb55be1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:27.259150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ade657-50a5-4481-9b13-c28e1bb55be1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:27.261259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:27.269354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5087a-fa4d-4229-aa5c-0ed65b4e3818 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:27.270103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5087a-fa4d-4229-aa5c-0ed65b4e3818 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:27.270262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5087a-fa4d-4229-aa5c-0ed65b4e3818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:27.353928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c0c5087a-fa4d-4229-aa5c-0ed65b4e3818 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:27.355175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c0c5087a-fa4d-4229-aa5c-0ed65b4e3818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:27.387200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:27.462739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:27.613399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ade657-50a5-4481-9b13-c28e1bb55be1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:27.614829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-02ade657-50a5-4481-9b13-c28e1bb55be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:27.770273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ade657-50a5-4481-9b13-c28e1bb55be1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:27.812991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:27.897960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e70f0-ab1d-45ac-9fe3-4016e47822b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:27.898534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e70f0-ab1d-45ac-9fe3-4016e47822b9 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:27.898813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e70f0-ab1d-45ac-9fe3-4016e47822b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:27.899083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e70f0-ab1d-45ac-9fe3-4016e47822b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:27.900710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f0e70f0-ab1d-45ac-9fe3-4016e47822b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:27.995196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e70f0-ab1d-45ac-9fe3-4016e47822b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:27.996859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e70f0-ab1d-45ac-9fe3-4016e47822b9 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:28.015358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 703/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:28.886442 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a65c2f-4f36-427f-9412-f5d2c9961eb4 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:28.887117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a65c2f-4f36-427f-9412-f5d2c9961eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:28.887117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a65c2f-4f36-427f-9412-f5d2c9961eb4 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:28.888405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:28.893800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcff226-094a-49b9-a93a-646c5d114a82 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:28.894227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcff226-094a-49b9-a93a-646c5d114a82 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:28.894359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcff226-094a-49b9-a93a-646c5d114a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:28.952195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:28.974734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7fcff226-094a-49b9-a93a-646c5d114a82 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:28.975938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7fcff226-094a-49b9-a93a-646c5d114a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:29.082477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:29.251763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a65c2f-4f36-427f-9412-f5d2c9961eb4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:29.253462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a1a65c2f-4f36-427f-9412-f5d2c9961eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:29.409958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a65c2f-4f36-427f-9412-f5d2c9961eb4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:29.422574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:29.537553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133156c-1310-496e-9683-d92d6e20bff6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:29.538102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133156c-1310-496e-9683-d92d6e20bff6 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:29.538352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133156c-1310-496e-9683-d92d6e20bff6 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:29.538651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133156c-1310-496e-9683-d92d6e20bff6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:29.540283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e133156c-1310-496e-9683-d92d6e20bff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:29.632359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133156c-1310-496e-9683-d92d6e20bff6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:29.634022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133156c-1310-496e-9683-d92d6e20bff6 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:29.652315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 718/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:30.501616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de58e8-998a-49df-a3f5-b10cc221cecc None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:30.502187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de58e8-998a-49df-a3f5-b10cc221cecc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:30.502317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de58e8-998a-49df-a3f5-b10cc221cecc None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:30.504481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:30.512327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090083d-662e-47d9-bb50-5cff3b631b3b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:30.512723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090083d-662e-47d9-bb50-5cff3b631b3b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:30.512872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090083d-662e-47d9-bb50-5cff3b631b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:30.531894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:30.593857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a090083d-662e-47d9-bb50-5cff3b631b3b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:30.595060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a090083d-662e-47d9-bb50-5cff3b631b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:30.694757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:30.861117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de58e8-998a-49df-a3f5-b10cc221cecc None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:30.863550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e9de58e8-998a-49df-a3f5-b10cc221cecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:30.984621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:31.022194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de58e8-998a-49df-a3f5-b10cc221cecc None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:31.148421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b62232-e86a-4735-b731-d7d9e9e0f3aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:31.149201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b62232-e86a-4735-b731-d7d9e9e0f3aa service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:31.149603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b62232-e86a-4735-b731-d7d9e9e0f3aa service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:31.149980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b62232-e86a-4735-b731-d7d9e9e0f3aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:31.153211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-41b62232-e86a-4735-b731-d7d9e9e0f3aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:31.244269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:31.260418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b62232-e86a-4735-b731-d7d9e9e0f3aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:31.262634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b62232-e86a-4735-b731-d7d9e9e0f3aa service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:31.281165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 706/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:32.141652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c6789-0248-4226-a102-ef4430b75ade None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:32.142249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c6789-0248-4226-a102-ef4430b75ade None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:32.142249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c6789-0248-4226-a102-ef4430b75ade None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:32.143625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:32.149810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb96eed-5219-41c7-aeed-883dcad9bbc4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:32.150598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb96eed-5219-41c7-aeed-883dcad9bbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:32.150746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb96eed-5219-41c7-aeed-883dcad9bbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:32.236771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4eb96eed-5219-41c7-aeed-883dcad9bbc4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:32.238017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4eb96eed-5219-41c7-aeed-883dcad9bbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:32.358237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:32.511955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c6789-0248-4226-a102-ef4430b75ade None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:32.514394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2f5c6789-0248-4226-a102-ef4430b75ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:32.617700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:32.683998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c6789-0248-4226-a102-ef4430b75ade None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:32.812538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d53f2-19ab-4a71-8e63-e21959980955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:32.813594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d53f2-19ab-4a71-8e63-e21959980955 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:32.813871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d53f2-19ab-4a71-8e63-e21959980955 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:32.814141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d53f2-19ab-4a71-8e63-e21959980955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:32.815783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-748d53f2-19ab-4a71-8e63-e21959980955 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:32.876530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:32.919010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d53f2-19ab-4a71-8e63-e21959980955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:32.920630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d53f2-19ab-4a71-8e63-e21959980955 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:32.939874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 721/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:33.742892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479212fb-7336-4023-94be-a143297ae9ec None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:33.743810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479212fb-7336-4023-94be-a143297ae9ec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:33.743810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479212fb-7336-4023-94be-a143297ae9ec None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:33.745928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:33.751177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a01cef-5513-422c-839c-e26aa8b9c0e5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:33.751608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a01cef-5513-422c-839c-e26aa8b9c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:33.751711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a01cef-5513-422c-839c-e26aa8b9c0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:33.832585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-05a01cef-5513-422c-839c-e26aa8b9c0e5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:33.833823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-05a01cef-5513-422c-839c-e26aa8b9c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:33.947513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:34.099616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479212fb-7336-4023-94be-a143297ae9ec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:34.101702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-479212fb-7336-4023-94be-a143297ae9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:34.162356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:34.276201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479212fb-7336-4023-94be-a143297ae9ec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:34.397485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc86958-7839-40d0-9ea2-58b07a6b4bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:34.398055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc86958-7839-40d0-9ea2-58b07a6b4bd4 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:34.398340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc86958-7839-40d0-9ea2-58b07a6b4bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:34.398619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc86958-7839-40d0-9ea2-58b07a6b4bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:34.400265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cbc86958-7839-40d0-9ea2-58b07a6b4bd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:34.407733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:34.518904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc86958-7839-40d0-9ea2-58b07a6b4bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:34.520525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc86958-7839-40d0-9ea2-58b07a6b4bd4 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:34.546165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 709/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:35.351033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d4ef-3f99-448f-a007-6dd908a69a95 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:35.351939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d4ef-3f99-448f-a007-6dd908a69a95 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:35.351939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d4ef-3f99-448f-a007-6dd908a69a95 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:35.354169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:35.362436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64656df-9eed-4689-aeb1-c29d3284d348 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:35.362873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64656df-9eed-4689-aeb1-c29d3284d348 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:35.363041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64656df-9eed-4689-aeb1-c29d3284d348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:35.446707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f64656df-9eed-4689-aeb1-c29d3284d348 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:35.447924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f64656df-9eed-4689-aeb1-c29d3284d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:35.530083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:35.560342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:35.722175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1d4ef-3f99-448f-a007-6dd908a69a95 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:35.724647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-dab1d4ef-3f99-448f-a007-6dd908a69a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:35.744964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:35.892680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1d4ef-3f99-448f-a007-6dd908a69a95 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:35.982465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:36.010658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e04dd-81e7-4132-aaaf-e0d08ff0e693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:36.011322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e04dd-81e7-4132-aaaf-e0d08ff0e693 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:36.011600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e04dd-81e7-4132-aaaf-e0d08ff0e693 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:36.011872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e04dd-81e7-4132-aaaf-e0d08ff0e693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:36.013598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9e1e04dd-81e7-4132-aaaf-e0d08ff0e693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:36.118800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e04dd-81e7-4132-aaaf-e0d08ff0e693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:36.120389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e04dd-81e7-4132-aaaf-e0d08ff0e693 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:36.138655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 724/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:36.952394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e859fe7-5816-486f-820e-42241d9324b0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:36.953213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e859fe7-5816-486f-820e-42241d9324b0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:36.953213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e859fe7-5816-486f-820e-42241d9324b0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:36.954877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:36.961367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcabafe-8f69-41f3-8f0b-fda269250a3d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:36.961836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcabafe-8f69-41f3-8f0b-fda269250a3d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:36.961932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcabafe-8f69-41f3-8f0b-fda269250a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:37.043571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bbcabafe-8f69-41f3-8f0b-fda269250a3d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:37.044816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bbcabafe-8f69-41f3-8f0b-fda269250a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:37.088059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:37.147485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:37.328950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e859fe7-5816-486f-820e-42241d9324b0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:37.330540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6e859fe7-5816-486f-820e-42241d9324b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:37.501908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e859fe7-5816-486f-820e-42241d9324b0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:37.559614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:37.625011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb1b7f-82d2-4035-9e1d-8565678db9e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:37.625718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1b7f-82d2-4035-9e1d-8565678db9e3 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:37.625998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1b7f-82d2-4035-9e1d-8565678db9e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:37.626283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb1b7f-82d2-4035-9e1d-8565678db9e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:37.627939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-18fb1b7f-82d2-4035-9e1d-8565678db9e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:37.722599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb1b7f-82d2-4035-9e1d-8565678db9e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:37.724190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb1b7f-82d2-4035-9e1d-8565678db9e3 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:37.749395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 712/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:38.554267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac942c4f-91ad-4f23-9e83-886d5d970d71 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:38.554917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac942c4f-91ad-4f23-9e83-886d5d970d71 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:38.554917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac942c4f-91ad-4f23-9e83-886d5d970d71 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:38.556219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:38.562477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770bc2e-23b3-48dd-b578-4205645f998e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:38.563582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770bc2e-23b3-48dd-b578-4205645f998e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:38.564003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770bc2e-23b3-48dd-b578-4205645f998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:38.646364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1770bc2e-23b3-48dd-b578-4205645f998e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:38.647571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1770bc2e-23b3-48dd-b578-4205645f998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:38.649653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:38.748883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:38.902043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac942c4f-91ad-4f23-9e83-886d5d970d71 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:38.903722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ac942c4f-91ad-4f23-9e83-886d5d970d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:39.080052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac942c4f-91ad-4f23-9e83-886d5d970d71 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:39.107959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:39.208527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc6267-1528-4a1a-8c87-0633a82ddb37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:39.209205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6267-1528-4a1a-8c87-0633a82ddb37 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:39.209481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6267-1528-4a1a-8c87-0633a82ddb37 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:39.209752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6267-1528-4a1a-8c87-0633a82ddb37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:39.211402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-febc6267-1528-4a1a-8c87-0633a82ddb37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:39.302690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc6267-1528-4a1a-8c87-0633a82ddb37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:39.304328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc6267-1528-4a1a-8c87-0633a82ddb37 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:39.327742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 727/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:41.231917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca059-cd49-4463-89b9-c4dac7c598fe None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:41.232466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca059-cd49-4463-89b9-c4dac7c598fe None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:41.232466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca059-cd49-4463-89b9-c4dac7c598fe None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:41.233966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:41.239356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2f76-73bd-4e7b-b918-344f356697f9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:41.239767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2f76-73bd-4e7b-b918-344f356697f9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:41.239914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2f76-73bd-4e7b-b918-344f356697f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:41.283238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:41.432920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4afa2f76-73bd-4e7b-b918-344f356697f9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:41.434192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4afa2f76-73bd-4e7b-b918-344f356697f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:41.536733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:41.575215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca059-cd49-4463-89b9-c4dac7c598fe None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:41.576750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-baeca059-cd49-4463-89b9-c4dac7c598fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:41.697003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca059-cd49-4463-89b9-c4dac7c598fe None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:41.730609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:41.780338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ac4a1-a9d2-4c2b-ad77-dd254c17fcd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:41.780986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ac4a1-a9d2-4c2b-ad77-dd254c17fcd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:41.781323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ac4a1-a9d2-4c2b-ad77-dd254c17fcd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:41.781593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ac4a1-a9d2-4c2b-ad77-dd254c17fcd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:41.783336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d66ac4a1-a9d2-4c2b-ad77-dd254c17fcd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:41.895687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ac4a1-a9d2-4c2b-ad77-dd254c17fcd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:41.897026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ac4a1-a9d2-4c2b-ad77-dd254c17fcd5 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:41.917029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 715/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:44.116872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07257-0e8d-48d1-9004-75088d0e0db9 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:44.117791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07257-0e8d-48d1-9004-75088d0e0db9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:44.117791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07257-0e8d-48d1-9004-75088d0e0db9 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:44.120032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:44.126652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4b1e5-7888-48ec-94a0-440b5443b532 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:44.127420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4b1e5-7888-48ec-94a0-440b5443b532 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:44.127607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4b1e5-7888-48ec-94a0-440b5443b532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:44.198999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fcd4b1e5-7888-48ec-94a0-440b5443b532 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:44.200222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fcd4b1e5-7888-48ec-94a0-440b5443b532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:44.214117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:44.309984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:44.348398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a07257-0e8d-48d1-9004-75088d0e0db9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:44.349997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-04a07257-0e8d-48d1-9004-75088d0e0db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:44.463172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a07257-0e8d-48d1-9004-75088d0e0db9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:44.476039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:44.544896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c548437-9432-42ac-8fd2-86ef833eb9c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:44.545607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c548437-9432-42ac-8fd2-86ef833eb9c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:44.545870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c548437-9432-42ac-8fd2-86ef833eb9c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:44.546138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c548437-9432-42ac-8fd2-86ef833eb9c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:44.548201 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1c548437-9432-42ac-8fd2-86ef833eb9c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:44.673411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c548437-9432-42ac-8fd2-86ef833eb9c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:44.674763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c548437-9432-42ac-8fd2-86ef833eb9c0 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:44.696110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 730/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:47.073164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a186-41ce-44aa-b705-113f01188769 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:47.074019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a186-41ce-44aa-b705-113f01188769 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:47.074019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a186-41ce-44aa-b705-113f01188769 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:47.076664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:47.085783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b67d2-e640-4a6b-a54c-e816c595dc3d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:47.086204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b67d2-e640-4a6b-a54c-e816c595dc3d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:47.086380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b67d2-e640-4a6b-a54c-e816c595dc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:47.136765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:47.157342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-340b67d2-e640-4a6b-a54c-e816c595dc3d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:47.158552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-340b67d2-e640-4a6b-a54c-e816c595dc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:47.267441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:47.307572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a186-41ce-44aa-b705-113f01188769 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:47.309961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cf61a186-41ce-44aa-b705-113f01188769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:47.430175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a186-41ce-44aa-b705-113f01188769 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:47.478503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:47.520214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1108516-f4ad-4889-a5e8-b642ed325b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:47.520796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108516-f4ad-4889-a5e8-b642ed325b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:47.521087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108516-f4ad-4889-a5e8-b642ed325b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:47.521379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108516-f4ad-4889-a5e8-b642ed325b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:47.523050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a1108516-f4ad-4889-a5e8-b642ed325b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:47.638519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108516-f4ad-4889-a5e8-b642ed325b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:47.639967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108516-f4ad-4889-a5e8-b642ed325b17 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:47.658808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 718/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:49.797907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b39f7c-82ae-45db-884e-095d9a917272 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:49.798539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b39f7c-82ae-45db-884e-095d9a917272 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:49.798539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b39f7c-82ae-45db-884e-095d9a917272 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:49.799832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:49.805604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb56c8-192f-489c-90e2-cb26156fb194 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:49.806078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb56c8-192f-489c-90e2-cb26156fb194 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:49.806199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb56c8-192f-489c-90e2-cb26156fb194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:49.874264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-24bb56c8-192f-489c-90e2-cb26156fb194 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:49.875453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-24bb56c8-192f-489c-90e2-cb26156fb194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:49.896745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:49.975418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:50.013816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b39f7c-82ae-45db-884e-095d9a917272 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:50.015375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-11b39f7c-82ae-45db-884e-095d9a917272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:50.131850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b39f7c-82ae-45db-884e-095d9a917272 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:50.153746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:50.211299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59888f8c-0a56-4c11-a984-926c0bd4e417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:50.211941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59888f8c-0a56-4c11-a984-926c0bd4e417 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:50.212224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59888f8c-0a56-4c11-a984-926c0bd4e417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:50.212496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59888f8c-0a56-4c11-a984-926c0bd4e417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:50.214182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-59888f8c-0a56-4c11-a984-926c0bd4e417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:50.347940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59888f8c-0a56-4c11-a984-926c0bd4e417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:50.349415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59888f8c-0a56-4c11-a984-926c0bd4e417 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:50.368457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 733/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:52.510223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872899f4-5cc3-485b-bf7e-a577d45ac7ef None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:52.510795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872899f4-5cc3-485b-bf7e-a577d45ac7ef None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:52.510795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872899f4-5cc3-485b-bf7e-a577d45ac7ef None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:52.512936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:52.519516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781c5fb-cf75-456c-bfde-60ad510bf269 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:52.520267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781c5fb-cf75-456c-bfde-60ad510bf269 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:52.520439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781c5fb-cf75-456c-bfde-60ad510bf269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:52.598104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:52.603121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d781c5fb-cf75-456c-bfde-60ad510bf269 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:52.605750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d781c5fb-cf75-456c-bfde-60ad510bf269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:52.707725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:52.754026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872899f4-5cc3-485b-bf7e-a577d45ac7ef None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:52.755559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-872899f4-5cc3-485b-bf7e-a577d45ac7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:52.878255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872899f4-5cc3-485b-bf7e-a577d45ac7ef None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:52.930444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:52.956493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2834f0c-3e97-4c62-aaad-8c56de252286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:52.957174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834f0c-3e97-4c62-aaad-8c56de252286 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:52.957458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834f0c-3e97-4c62-aaad-8c56de252286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:52.957730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834f0c-3e97-4c62-aaad-8c56de252286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:52.959356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e2834f0c-3e97-4c62-aaad-8c56de252286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:53.075740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2834f0c-3e97-4c62-aaad-8c56de252286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:53.077211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2834f0c-3e97-4c62-aaad-8c56de252286 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:53.098659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 721/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:55.236113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef13dc5-6c8b-468f-b932-e574dcdfac52 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:55.236706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef13dc5-6c8b-468f-b932-e574dcdfac52 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:55.236706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef13dc5-6c8b-468f-b932-e574dcdfac52 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:55.238269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:55.244915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59a36-2d44-465e-9ec2-ebd2187cd66f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:55.244915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59a36-2d44-465e-9ec2-ebd2187cd66f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:55.245395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a59a36-2d44-465e-9ec2-ebd2187cd66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:55.316364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-74a59a36-2d44-465e-9ec2-ebd2187cd66f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:55.317675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-74a59a36-2d44-465e-9ec2-ebd2187cd66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:55.348006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:55.421534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:55.464602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef13dc5-6c8b-468f-b932-e574dcdfac52 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:55.466641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8ef13dc5-6c8b-468f-b932-e574dcdfac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:55.589537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef13dc5-6c8b-468f-b932-e574dcdfac52 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:55.626096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:55.676735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d960df-dc1b-4e15-8948-bb6c5565acfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:55.677415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d960df-dc1b-4e15-8948-bb6c5565acfb service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:55.677696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d960df-dc1b-4e15-8948-bb6c5565acfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:55.677974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d960df-dc1b-4e15-8948-bb6c5565acfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:55.679625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d0d960df-dc1b-4e15-8948-bb6c5565acfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:55.803158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d960df-dc1b-4e15-8948-bb6c5565acfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:55.804593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d960df-dc1b-4e15-8948-bb6c5565acfb service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:55.823996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 736/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:06:57.882710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261d22-86bd-4ef4-b588-37113c412677 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:57.883330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261d22-86bd-4ef4-b588-37113c412677 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:57.883330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261d22-86bd-4ef4-b588-37113c412677 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:57.884654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:06:57.891028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fd7ef-8c2f-4c2f-b087-f025f1f72a3d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:57.891779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fd7ef-8c2f-4c2f-b087-f025f1f72a3d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:57.891969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fd7ef-8c2f-4c2f-b087-f025f1f72a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:57.970805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6f9fd7ef-8c2f-4c2f-b087-f025f1f72a3d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:06:57.972004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6f9fd7ef-8c2f-4c2f-b087-f025f1f72a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:57.980868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:58.075043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:06:58.115823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261d22-86bd-4ef4-b588-37113c412677 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:06:58.118259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b0261d22-86bd-4ef4-b588-37113c412677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:58.234210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261d22-86bd-4ef4-b588-37113c412677 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:06:58.305369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:06:58.325851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b1b20-9dd3-4063-8121-a8784a74fc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:06:58.326509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b1b20-9dd3-4063-8121-a8784a74fc43 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:06:58.326791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b1b20-9dd3-4063-8121-a8784a74fc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:06:58.327061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b1b20-9dd3-4063-8121-a8784a74fc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:06:58.328727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d35b1b20-9dd3-4063-8121-a8784a74fc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:06:58.446071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b1b20-9dd3-4063-8121-a8784a74fc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:06:58.447520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b1b20-9dd3-4063-8121-a8784a74fc43 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:06:58.466603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 724/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:00.604661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e895f3-4f01-456a-bb72-2dcc10e78be0 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:00.605239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e895f3-4f01-456a-bb72-2dcc10e78be0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:00.605239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e895f3-4f01-456a-bb72-2dcc10e78be0 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:00.607503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:07:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:00.613950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6a0b7-956c-4035-94f5-b3b1c828c0e9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:00.614715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6a0b7-956c-4035-94f5-b3b1c828c0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:00.614855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6a0b7-956c-4035-94f5-b3b1c828c0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:00.699554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2ef6a0b7-956c-4035-94f5-b3b1c828c0e9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:00.700828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2ef6a0b7-956c-4035-94f5-b3b1c828c0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:00.740095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:00.801576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:00.839704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e895f3-4f01-456a-bb72-2dcc10e78be0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:00.842195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b1e895f3-4f01-456a-bb72-2dcc10e78be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:00.961958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e895f3-4f01-456a-bb72-2dcc10e78be0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:01.005757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:01.044091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1509e-bad2-427a-9c10-0d72876eb234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:01.044745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1509e-bad2-427a-9c10-0d72876eb234 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:01.045025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1509e-bad2-427a-9c10-0d72876eb234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:01.045344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1509e-bad2-427a-9c10-0d72876eb234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:01.047022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-44d1509e-bad2-427a-9c10-0d72876eb234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:01.159621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1509e-bad2-427a-9c10-0d72876eb234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:01.161097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1509e-bad2-427a-9c10-0d72876eb234 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:01.180301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 739/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:03.224402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010a77c-6265-4d7a-a510-1db9aac67140 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:03.225022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010a77c-6265-4d7a-a510-1db9aac67140 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:03.225022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010a77c-6265-4d7a-a510-1db9aac67140 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:03.227128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:07:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:03.232868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fdfca-8631-4396-b2b7-712e14c566ab None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:03.233352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fdfca-8631-4396-b2b7-712e14c566ab None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:03.233457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fdfca-8631-4396-b2b7-712e14c566ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:03.299808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-628fdfca-8631-4396-b2b7-712e14c566ab None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:03.300999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-628fdfca-8631-4396-b2b7-712e14c566ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:03.319915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:03.407207 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:03.457836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010a77c-6265-4d7a-a510-1db9aac67140 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:03.459383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9010a77c-6265-4d7a-a510-1db9aac67140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:03.581290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010a77c-6265-4d7a-a510-1db9aac67140 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:03.647501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:03.654888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d960cf-e669-446b-93dc-b3945e83c493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:03.655541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d960cf-e669-446b-93dc-b3945e83c493 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:03.655816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d960cf-e669-446b-93dc-b3945e83c493 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:03.656086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d960cf-e669-446b-93dc-b3945e83c493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:03.657771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-51d960cf-e669-446b-93dc-b3945e83c493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:03.790970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d960cf-e669-446b-93dc-b3945e83c493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:03.793096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d960cf-e669-446b-93dc-b3945e83c493 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:03.813483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 727/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:05.863961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461395-d9e1-42b6-85d9-78a38b8713eb None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:05.864589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461395-d9e1-42b6-85d9-78a38b8713eb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:05.864589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461395-d9e1-42b6-85d9-78a38b8713eb None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:05.870245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:05.874415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5df6-549a-44d7-a3e7-d41983244555 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:05.874893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5df6-549a-44d7-a3e7-d41983244555 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:05.875108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5df6-549a-44d7-a3e7-d41983244555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:05.946142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f4cf5df6-549a-44d7-a3e7-d41983244555 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:05.947394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f4cf5df6-549a-44d7-a3e7-d41983244555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:05.998813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:06.054899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:06.101835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461395-d9e1-42b6-85d9-78a38b8713eb None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:06.103384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8461395-d9e1-42b6-85d9-78a38b8713eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:06.215800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461395-d9e1-42b6-85d9-78a38b8713eb None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:06.264122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:06.295502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea9361-8dbb-4c4b-9c52-1ee61a036a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:06.296153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9361-8dbb-4c4b-9c52-1ee61a036a6e service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:06.296433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9361-8dbb-4c4b-9c52-1ee61a036a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:06.296707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9361-8dbb-4c4b-9c52-1ee61a036a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:06.298374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-62ea9361-8dbb-4c4b-9c52-1ee61a036a6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:06.417389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea9361-8dbb-4c4b-9c52-1ee61a036a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:06.419506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea9361-8dbb-4c4b-9c52-1ee61a036a6e service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:06.440909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 742/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:16.052312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=22 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:16.438059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=66 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:26.281932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028a0a3-1ed4-4fcd-b746-6552778865fc None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:26.282809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028a0a3-1ed4-4fcd-b746-6552778865fc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:26.282809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028a0a3-1ed4-4fcd-b746-6552778865fc None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:26.284885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 729/1471] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:26.290230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca719ab9-2b36-4644-a79c-3a6f8d961c69 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:26.290677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca719ab9-2b36-4644-a79c-3a6f8d961c69 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:26.290782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca719ab9-2b36-4644-a79c-3a6f8d961c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:26.361560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ca719ab9-2b36-4644-a79c-3a6f8d961c69 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:26.362772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ca719ab9-2b36-4644-a79c-3a6f8d961c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:26.469922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 743/1472] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:26.558088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac774dd-84b7-41c6-b34d-691189882ff2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:26.558365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac774dd-84b7-41c6-b34d-691189882ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:26.558661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac774dd-84b7-41c6-b34d-691189882ff2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:26.561212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 730/1473] 10.222.0.25 () {54 vars in 962 bytes} [Tue Jul 12 07:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:26.567346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da650dc-daad-462a-b646-e8ad8bf56d21 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:26.567687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da650dc-daad-462a-b646-e8ad8bf56d21 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:26.568030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da650dc-daad-462a-b646-e8ad8bf56d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:26.658171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0da650dc-daad-462a-b646-e8ad8bf56d21 None None] MFA Rules not processed for user `366a7e7596d14b32a6ce62790a2e3dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:26.659368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0da650dc-daad-462a-b646-e8ad8bf56d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:26.680058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:26.739050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 744/1474] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Jul 12 07:07:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:26.748938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac774dd-84b7-41c6-b34d-691189882ff2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:26.751801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6ac774dd-84b7-41c6-b34d-691189882ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:26.870182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac774dd-84b7-41c6-b34d-691189882ff2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:26.947360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9294464-caba-4d60-a7d0-b0e6ae83bead service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366a7e7596d14b32a6ce62790a2e3dd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:26.948043 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9294464-caba-4d60-a7d0-b0e6ae83bead service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:26.948391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9294464-caba-4d60-a7d0-b0e6ae83bead service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:26.948630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9294464-caba-4d60-a7d0-b0e6ae83bead service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:26.950385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f9294464-caba-4d60-a7d0-b0e6ae83bead service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:27.008900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:27.068609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9294464-caba-4d60-a7d0-b0e6ae83bead service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:27.070081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9294464-caba-4d60-a7d0-b0e6ae83bead service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:27.090385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 731/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:29.202740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a976c1-c2ca-4380-bb45-8ce2c7edfb03 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:29.203459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a976c1-c2ca-4380-bb45-8ce2c7edfb03 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:29.203459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a976c1-c2ca-4380-bb45-8ce2c7edfb03 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:29.204700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 745/1476] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:29.213718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3aaf7-232b-4f35-adf8-555aa04ddff7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:29.214178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3aaf7-232b-4f35-adf8-555aa04ddff7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:29.214284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3aaf7-232b-4f35-adf8-555aa04ddff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:29.288716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-adf3aaf7-232b-4f35-adf8-555aa04ddff7 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:29.289982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-adf3aaf7-232b-4f35-adf8-555aa04ddff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:29.390666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 732/1477] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:29.415544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a976c1-c2ca-4380-bb45-8ce2c7edfb03 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:29.418088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-11a976c1-c2ca-4380-bb45-8ce2c7edfb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:29.561715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a976c1-c2ca-4380-bb45-8ce2c7edfb03 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:29.610581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:29.659211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f22edd-aa28-4034-be4b-70510c28c200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:29.659876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f22edd-aa28-4034-be4b-70510c28c200 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:29.660153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f22edd-aa28-4034-be4b-70510c28c200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:29.660420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f22edd-aa28-4034-be4b-70510c28c200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:29.661188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f22edd-aa28-4034-be4b-70510c28c200 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:29.662928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-c4f22edd-aa28-4034-be4b-70510c28c200 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:07:29.662928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:07:29.662928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:07:29.664060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f22edd-aa28-4034-be4b-70510c28c200 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:29.690050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 746/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 07:07:29 2022] GET /identity/v3/projects => generated 2553 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:07:31.732460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b002f1-b439-49da-98c8-fc40ca68a097 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:31.733435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b002f1-b439-49da-98c8-fc40ca68a097 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:31.733435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b002f1-b439-49da-98c8-fc40ca68a097 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:31.735586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 733/1479] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:07:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:31.742148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5524-f320-4dec-9923-4233cb044a4f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:31.742909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5524-f320-4dec-9923-4233cb044a4f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:31.743075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5524-f320-4dec-9923-4233cb044a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:31.820227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-da1f5524-f320-4dec-9923-4233cb044a4f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:31.821458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-da1f5524-f320-4dec-9923-4233cb044a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:31.923002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 747/1480] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:31.974683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b002f1-b439-49da-98c8-fc40ca68a097 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:31.976298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-77b002f1-b439-49da-98c8-fc40ca68a097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:31.990862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:32.132493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b002f1-b439-49da-98c8-fc40ca68a097 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:32.219676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:32.254638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e26d0-16a1-4f23-990b-095375d54f6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:32.255222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e26d0-16a1-4f23-990b-095375d54f6d service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:32.255495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e26d0-16a1-4f23-990b-095375d54f6d service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:32.255763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e26d0-16a1-4f23-990b-095375d54f6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:32.257434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8f4e26d0-16a1-4f23-990b-095375d54f6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:32.351058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e26d0-16a1-4f23-990b-095375d54f6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:32.352653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e26d0-16a1-4f23-990b-095375d54f6d service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:32.380791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 734/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:34.605293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e786df-077a-4c69-be60-f6aba6707dc7 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:34.605910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e786df-077a-4c69-be60-f6aba6707dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:34.605910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e786df-077a-4c69-be60-f6aba6707dc7 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:34.607258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 748/1482] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:34.613935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efd8e2-659e-4b4f-a54a-a977af242e19 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:34.614672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efd8e2-659e-4b4f-a54a-a977af242e19 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:34.614848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efd8e2-659e-4b4f-a54a-a977af242e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:34.690218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-87efd8e2-659e-4b4f-a54a-a977af242e19 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:34.691414 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-87efd8e2-659e-4b4f-a54a-a977af242e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:34.738232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:34.797139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 735/1483] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:34.833796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e786df-077a-4c69-be60-f6aba6707dc7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:34.836191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b9e786df-077a-4c69-be60-f6aba6707dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:34.862423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:34.954630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e786df-077a-4c69-be60-f6aba6707dc7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:35.032890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74622-4693-43e4-9742-49fa84717608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:35.033550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74622-4693-43e4-9742-49fa84717608 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:35.033829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74622-4693-43e4-9742-49fa84717608 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:35.034100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74622-4693-43e4-9742-49fa84717608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:35.035726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-47d74622-4693-43e4-9742-49fa84717608 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:35.117687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:35.156147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d74622-4693-43e4-9742-49fa84717608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:35.157637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d74622-4693-43e4-9742-49fa84717608 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:35.177100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 749/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:37.259093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128058b-5aca-4734-bd74-84f98307d8b4 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:37.259659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128058b-5aca-4734-bd74-84f98307d8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:37.259659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128058b-5aca-4734-bd74-84f98307d8b4 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:37.261677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 736/1485] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:37.268564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81f62-8260-457c-ad15-67d5d305e617 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:37.269369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81f62-8260-457c-ad15-67d5d305e617 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:37.269369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b81f62-8260-457c-ad15-67d5d305e617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:37.351489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-79b81f62-8260-457c-ad15-67d5d305e617 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:37.352721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-79b81f62-8260-457c-ad15-67d5d305e617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:37.466520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 750/1486] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:37.507338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128058b-5aca-4734-bd74-84f98307d8b4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:37.508895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3128058b-5aca-4734-bd74-84f98307d8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:37.631784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128058b-5aca-4734-bd74-84f98307d8b4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:37.708953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:37.710841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b3d3e-f0f5-4624-8e36-3f6061543610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:37.711552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b3d3e-f0f5-4624-8e36-3f6061543610 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:37.711894 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b3d3e-f0f5-4624-8e36-3f6061543610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:37.712179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b3d3e-f0f5-4624-8e36-3f6061543610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:37.714398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-455b3d3e-f0f5-4624-8e36-3f6061543610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:37.826133 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b3d3e-f0f5-4624-8e36-3f6061543610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:37.827568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b3d3e-f0f5-4624-8e36-3f6061543610 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:37.846481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 737/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:40.013604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7c60-078d-4eb6-9bf9-324235fdcc2f None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:40.014170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7c60-078d-4eb6-9bf9-324235fdcc2f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:40.014170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7c60-078d-4eb6-9bf9-324235fdcc2f None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:40.015557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 751/1488] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:40.024426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b6612-a15e-4d02-9d1d-d17d05819741 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:40.024887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b6612-a15e-4d02-9d1d-d17d05819741 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:40.024990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b6612-a15e-4d02-9d1d-d17d05819741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:40.101118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4d9b6612-a15e-4d02-9d1d-d17d05819741 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:40.102261 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4d9b6612-a15e-4d02-9d1d-d17d05819741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:40.162856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:40.211725 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 738/1489] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:40.256915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7c60-078d-4eb6-9bf9-324235fdcc2f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:40.259879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bcce7c60-078d-4eb6-9bf9-324235fdcc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:40.307928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:40.381613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7c60-078d-4eb6-9bf9-324235fdcc2f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:40.465139 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a5d20-9468-4aaf-b472-76244eec244a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:40.465793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a5d20-9468-4aaf-b472-76244eec244a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:40.466074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a5d20-9468-4aaf-b472-76244eec244a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:40.466336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a5d20-9468-4aaf-b472-76244eec244a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:40.467989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d04a5d20-9468-4aaf-b472-76244eec244a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:40.555440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:40.594397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a5d20-9468-4aaf-b472-76244eec244a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:40.595865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a5d20-9468-4aaf-b472-76244eec244a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:40.615508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 752/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:42.838911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570db9-763c-43b7-9d9a-d22ad0f4004e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:42.839834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570db9-763c-43b7-9d9a-d22ad0f4004e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:42.840222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570db9-763c-43b7-9d9a-d22ad0f4004e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:42.842669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 739/1491] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 07:07:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:07:42.848970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98cbef-7c08-42c8-b290-6eea0a3192d2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:42.849740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98cbef-7c08-42c8-b290-6eea0a3192d2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:42.849884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98cbef-7c08-42c8-b290-6eea0a3192d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:42.924771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ba98cbef-7c08-42c8-b290-6eea0a3192d2 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:07:42.926012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ba98cbef-7c08-42c8-b290-6eea0a3192d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:43.029866 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 753/1492] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:07:43.068990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33570db9-763c-43b7-9d9a-d22ad0f4004e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:07:43.070613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-33570db9-763c-43b7-9d9a-d22ad0f4004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:43.188083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33570db9-763c-43b7-9d9a-d22ad0f4004e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:07:43.275886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ade16b-993c-47bd-92a0-d5bed141ed7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:07:43.276545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ade16b-993c-47bd-92a0-d5bed141ed7a service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:07:43.276944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ade16b-993c-47bd-92a0-d5bed141ed7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:07:43.277288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ade16b-993c-47bd-92a0-d5bed141ed7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:07:43.278954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d8ade16b-993c-47bd-92a0-d5bed141ed7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:07:43.288351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:43.390113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ade16b-993c-47bd-92a0-d5bed141ed7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:07:43.391556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ade16b-993c-47bd-92a0-d5bed141ed7a service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:07:43.410249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 740/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:07:53.027981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=82 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:07:53.407714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=50 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:16.355328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:16.355931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:16.355931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:16.437321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:09:16.438553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:16.541915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 754/1494] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:09:16.551381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540239c-f279-4727-9790-eb283f740422 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:16.552152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540239c-f279-4727-9790-eb283f740422 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:16.552258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540239c-f279-4727-9790-eb283f740422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:16.655883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6540239c-f279-4727-9790-eb283f740422 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:09:16.657024 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6540239c-f279-4727-9790-eb283f740422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:16.761532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 741/1495] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:09:16.769178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:16.771640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:16.787474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc064694-9c58-42ae-a68c-d13ccdf53855 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:16.858192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:16.897977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:16.898644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:16.898915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:16.899188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:16.899905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:09:16.900208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:16.901321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:16.901321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:09:16.901321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:09:16.902304 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:16.914312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 755/1496] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:09:16.923512 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540239c-f279-4727-9790-eb283f740422 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:16.934420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540239c-f279-4727-9790-eb283f740422 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:17.004225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:17.052715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:17.053709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:17.054114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:17.054510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:17.055564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:17.058669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:17.058669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 12 07:09:17.058669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: Jul 12 07:09:17.060243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:17.124300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 742/1497] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:09:16 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:09:17.128746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:17.130339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:17.165829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273aa6e-adbd-4850-bf4c-2e01ab4c0c14 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:17.267838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:17.268474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:17.268788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:17.269084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:17.269756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:17.270838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:17.270838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:09:17.270838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:09:17.272035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:17.315884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 756/1498] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:09:17 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:09:17.321030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:17.323539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:17.387061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27a7f6-ee78-45f7-8165-82b0707923a6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:17.478300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:17.490548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:17.491234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:17.491501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:17.491774 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:17.492511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:17.493582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:17.493582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 12 07:09:17.493582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: Jul 12 07:09:17.494474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:17.507222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 743/1499] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:09:17.511600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:17.513242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:17.524966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fdc3e-778c-4707-9104-44579179368d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:17.554013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:17.639486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc2cc1-6f9b-4ceb-81d9-6c0955a4e79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:17.640165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2cc1-6f9b-4ceb-81d9-6c0955a4e79a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:17.640451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2cc1-6f9b-4ceb-81d9-6c0955a4e79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:17.640720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2cc1-6f9b-4ceb-81d9-6c0955a4e79a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:17.641487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc2cc1-6f9b-4ceb-81d9-6c0955a4e79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:17.642531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-64dc2cc1-6f9b-4ceb-81d9-6c0955a4e79a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:17.642531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:09:17.642531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:09:17.643421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc2cc1-6f9b-4ceb-81d9-6c0955a4e79a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:17.656421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 757/1500] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:09:17.661164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:17.662714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:17.700572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e441e1f-d00f-4450-9023-8e4f2107a7fe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:17.819614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9abbb8-649d-4f31-9f95-bf0222ee5be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:17.820424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9abbb8-649d-4f31-9f95-bf0222ee5be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:17.820711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9abbb8-649d-4f31-9f95-bf0222ee5be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:17.820953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9abbb8-649d-4f31-9f95-bf0222ee5be2 admin admin] PATH_INFO: `/v3/projects/abee6c469c8f45998b8826413a2c226c/users/14373fe10c844efbb3ec8b26ed432e0b/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:17.848667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9abbb8-649d-4f31-9f95-bf0222ee5be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abee6c469c8f45998b8826413a2c226c, user_id=14373fe10c844efbb3ec8b26ed432e0b, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:17.849928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-ab9abbb8-649d-4f31-9f95-bf0222ee5be2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:17.849928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 12 07:09:17.849928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: Jul 12 07:09:17.852739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9abbb8-649d-4f31-9f95-bf0222ee5be2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:17.885486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 744/1501] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:09:17 2022] PUT /identity/v3/projects/abee6c469c8f45998b8826413a2c226c/users/14373fe10c844efbb3ec8b26ed432e0b/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:09:17.899333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab3a2-fabd-44b5-8e7d-b2e09ef8a966 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:17.899592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab3a2-fabd-44b5-8e7d-b2e09ef8a966 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:17.899898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab3a2-fabd-44b5-8e7d-b2e09ef8a966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:17.968815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b9aab3a2-fabd-44b5-8e7d-b2e09ef8a966 None None] MFA Rules not processed for user `14373fe10c844efbb3ec8b26ed432e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:09:17.970109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b9aab3a2-fabd-44b5-8e7d-b2e09ef8a966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:18.081974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 758/1502] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:09:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:09:18.098814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6d4c-37f7-4100-922b-3e30203e97af None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:18.099280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6d4c-37f7-4100-922b-3e30203e97af None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:18.099774 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6d4c-37f7-4100-922b-3e30203e97af None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:18.102027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 745/1503] 10.222.0.25 () {54 vars in 963 bytes} [Tue Jul 12 07:09:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:09:18.108594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11e1f1-381b-4c46-bcd1-cbf1109135f2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:18.108879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11e1f1-381b-4c46-bcd1-cbf1109135f2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:18.109187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11e1f1-381b-4c46-bcd1-cbf1109135f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:18.147217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:18.200927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2c11e1f1-381b-4c46-bcd1-cbf1109135f2 None None] MFA Rules not processed for user `b4292042e79c45438b5964710359cd5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:09:18.202152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2c11e1f1-381b-4c46-bcd1-cbf1109135f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:18.308398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 759/1504] 10.222.0.25 () {58 vars in 1060 bytes} [Tue Jul 12 07:09:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:09:18.317446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a6d4c-37f7-4100-922b-3e30203e97af None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:18.319528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-016a6d4c-37f7-4100-922b-3e30203e97af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:18.397969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:18.475852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a6d4c-37f7-4100-922b-3e30203e97af None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:18.568672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:18.569406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:18.569686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:18.569960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:18.571705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:18.704274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:18.705844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:18.709412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:18.727373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 746/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:09:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:09:18.776212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c314ce-98dd-4570-8622-8c31735f33aa None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:18.776692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c314ce-98dd-4570-8622-8c31735f33aa None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:18.776788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c314ce-98dd-4570-8622-8c31735f33aa None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:18.783266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 760/1506] 10.222.0.25 () {52 vars in 829 bytes} [Tue Jul 12 07:09:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:09:18.787188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:18.788420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:18.913028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385f0c5-ecc7-4b26-a6a3-9af2dc387c68 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:19.024091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:19.024972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:19.025279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:19.025544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] PATH_INFO: `/v3/users/14373fe10c844efbb3ec8b26ed432e0b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:19.038418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=14373fe10c844efbb3ec8b26ed432e0b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:19.039447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:19.039447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 12 07:09:19.039447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: Jul 12 07:09:19.040651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:19.050985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:19.109798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14373fe10c844efbb3ec8b26ed432e0b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:19.110230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14373fe10c844efbb3ec8b26ed432e0b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:19.130380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14373fe10c844efbb3ec8b26ed432e0b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:19.130935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14373fe10c844efbb3ec8b26ed432e0b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:19.144998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14373fe10c844efbb3ec8b26ed432e0b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:19.161782 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 747/1507] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:09:18 2022] DELETE /identity/v3/users/14373fe10c844efbb3ec8b26ed432e0b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:09:19.166419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66826850-0292-43e3-a242-067ad5df61ad None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:19.166709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66826850-0292-43e3-a242-067ad5df61ad None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:19.166985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66826850-0292-43e3-a242-067ad5df61ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:19.225774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-66826850-0292-43e3-a242-067ad5df61ad None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:09:19.226990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-66826850-0292-43e3-a242-067ad5df61ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:19.304770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:19.317672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 761/1508] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:09:19.329945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:19.331172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:19.439534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad9c2a-da69-4a66-bcde-7eb573a39a9b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:19.496792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:19.518073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb1a32-be19-4b57-aa38-2b91dbe0c007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:19.518726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb1a32-be19-4b57-aa38-2b91dbe0c007 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:19.519009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb1a32-be19-4b57-aa38-2b91dbe0c007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:19.519283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb1a32-be19-4b57-aa38-2b91dbe0c007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:19.520912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f8cb1a32-be19-4b57-aa38-2b91dbe0c007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:19.632769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb1a32-be19-4b57-aa38-2b91dbe0c007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:19.634163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb1a32-be19-4b57-aa38-2b91dbe0c007 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:19.654584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 748/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:09:20.004756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66826850-0292-43e3-a242-067ad5df61ad None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:09:20.006126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-66826850-0292-43e3-a242-067ad5df61ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:09:20.018947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66826850-0292-43e3-a242-067ad5df61ad None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:09:20.117516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:09:20.118391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:09:20.118669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:09:20.118933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] PATH_INFO: `/v3/projects/abee6c469c8f45998b8826413a2c226c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:09:20.129518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=abee6c469c8f45998b8826413a2c226c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:09:20.130555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:09:20.130555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:09:20.130555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:09:20.131736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:09:20.194664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abee6c469c8f45998b8826413a2c226c', 'request_id': 'req-f4112956-9b00-43cc-98ad-1e510fe199bf'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:20.208626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abee6c469c8f45998b8826413a2c226c', 'request_id': 'req-f4112956-9b00-43cc-98ad-1e510fe199bf'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:20.215111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abee6c469c8f45998b8826413a2c226c', 'request_id': 'req-f4112956-9b00-43cc-98ad-1e510fe199bf'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:20.242080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] The token cache is being invalidate because project abee6c469c8f45998b8826413a2c226c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:09:20.242365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:20.242780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f4112956-9b00-43cc-98ad-1e510fe199bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:09:20.244252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 762/1510] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:09:20 2022] DELETE /identity/v3/projects/abee6c469c8f45998b8826413a2c226c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:09:29.651699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=62 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:09:30.240174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:10:59.158660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954b6d-48ec-4f84-8396-8d2246b905be None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:10:59.159338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954b6d-48ec-4f84-8396-8d2246b905be None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:10:59.159338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954b6d-48ec-4f84-8396-8d2246b905be None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:10:59.161069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:10:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:10:59.168462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae51c5-77f0-4600-be7e-9d7886cf822d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:10:59.168926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae51c5-77f0-4600-be7e-9d7886cf822d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:10:59.169033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae51c5-77f0-4600-be7e-9d7886cf822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:10:59.256624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-afae51c5-77f0-4600-be7e-9d7886cf822d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:10:59.257754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-afae51c5-77f0-4600-be7e-9d7886cf822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:10:59.387685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:10:59.546891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86954b6d-48ec-4f84-8396-8d2246b905be None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:10:59.549642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-86954b6d-48ec-4f84-8396-8d2246b905be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:10:59.677041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86954b6d-48ec-4f84-8396-8d2246b905be None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:10:59.683215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c18879-499a-4a73-b407-061a63154410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:10:59.683884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c18879-499a-4a73-b407-061a63154410 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:10:59.684164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c18879-499a-4a73-b407-061a63154410 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:10:59.684443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c18879-499a-4a73-b407-061a63154410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:10:59.692974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c18879-499a-4a73-b407-061a63154410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:10:59.694553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c18879-499a-4a73-b407-061a63154410 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:10:59.704112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 750/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:00.535173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e654c7-19fb-49a4-bb14-9681120edd16 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:00.536108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e654c7-19fb-49a4-bb14-9681120edd16 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:00.536108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e654c7-19fb-49a4-bb14-9681120edd16 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:00.538428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:11:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:00.544704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76409b58-b42c-4fe2-b081-e23cdd5852b0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:00.545549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76409b58-b42c-4fe2-b081-e23cdd5852b0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:00.545726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76409b58-b42c-4fe2-b081-e23cdd5852b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:00.569366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-76409b58-b42c-4fe2-b081-e23cdd5852b0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:00.570573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-76409b58-b42c-4fe2-b081-e23cdd5852b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:00.579280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:00.733688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e654c7-19fb-49a4-bb14-9681120edd16 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:00.754133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e654c7-19fb-49a4-bb14-9681120edd16 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:00.760963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9de6d-9614-4086-8916-033a4b4c3ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:00.761632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9de6d-9614-4086-8916-033a4b4c3ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:00.761912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9de6d-9614-4086-8916-033a4b4c3ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:00.762185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9de6d-9614-4086-8916-033a4b4c3ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:00.773864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9de6d-9614-4086-8916-033a4b4c3ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:00.776449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9de6d-9614-4086-8916-033a4b4c3ea2 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:00.790573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 765/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:00.936411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020f305-e8a1-48d2-a97d-c055eba4b12b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:00.936667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020f305-e8a1-48d2-a97d-c055eba4b12b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:00.936954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020f305-e8a1-48d2-a97d-c055eba4b12b None None] PATH_INFO: `/v3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:00.938340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 752/1517] 10.222.0.40 () {54 vars in 925 bytes} [Tue Jul 12 07:11:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:11:00.970883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eb1a8-d12d-412d-867b-b0d785f0c570 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:00.971177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eb1a8-d12d-412d-867b-b0d785f0c570 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:00.971450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eb1a8-d12d-412d-867b-b0d785f0c570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:01.034960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-637eb1a8-d12d-412d-867b-b0d785f0c570 None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:01.036172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-637eb1a8-d12d-412d-867b-b0d785f0c570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:01.055557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:11:01.119464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 766/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Tue Jul 12 07:11:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:10.208721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c068-1b51-4339-88b4-4cda58c652ad None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:10.209361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c068-1b51-4339-88b4-4cda58c652ad None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:10.209361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c068-1b51-4339-88b4-4cda58c652ad None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:10.210685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 753/1519] 10.222.0.25 () {54 vars in 960 bytes} [Tue Jul 12 07:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:10.215996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ed41f-2553-47ec-8028-e21fd02eb48c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:10.216512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ed41f-2553-47ec-8028-e21fd02eb48c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:10.216512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ed41f-2553-47ec-8028-e21fd02eb48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:10.236923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b60ed41f-2553-47ec-8028-e21fd02eb48c None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:10.238165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b60ed41f-2553-47ec-8028-e21fd02eb48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:10.248295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 767/1520] 10.222.0.25 () {58 vars in 1057 bytes} [Tue Jul 12 07:11:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:10.256368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c068-1b51-4339-88b4-4cda58c652ad None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:10.271540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c068-1b51-4339-88b4-4cda58c652ad None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:10.277801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8938cde-575b-4765-a67d-092ffd4250bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:10.278444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8938cde-575b-4765-a67d-092ffd4250bf service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:10.278728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8938cde-575b-4765-a67d-092ffd4250bf service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:10.279000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8938cde-575b-4765-a67d-092ffd4250bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:10.288094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8938cde-575b-4765-a67d-092ffd4250bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:10.289747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8938cde-575b-4765-a67d-092ffd4250bf service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:10.299676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 754/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:12.045841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf0b0-b61a-4f71-85e3-5e6fa0f35120 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:12.046501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf0b0-b61a-4f71-85e3-5e6fa0f35120 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:12.046501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf0b0-b61a-4f71-85e3-5e6fa0f35120 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:12.047872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:12.054207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45028cff-1a76-4416-aa3a-c92c8646abe3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:12.054958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45028cff-1a76-4416-aa3a-c92c8646abe3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:12.055110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45028cff-1a76-4416-aa3a-c92c8646abe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:12.085859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-45028cff-1a76-4416-aa3a-c92c8646abe3 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:12.087057 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-45028cff-1a76-4416-aa3a-c92c8646abe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:12.095625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:12.249178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf0b0-b61a-4f71-85e3-5e6fa0f35120 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:12.270614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf0b0-b61a-4f71-85e3-5e6fa0f35120 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:12.277010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd784fa-e496-4f8e-983e-ddfcbb990178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:12.277682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd784fa-e496-4f8e-983e-ddfcbb990178 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:12.277961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd784fa-e496-4f8e-983e-ddfcbb990178 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:12.278235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd784fa-e496-4f8e-983e-ddfcbb990178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:12.286944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd784fa-e496-4f8e-983e-ddfcbb990178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:12.288538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd784fa-e496-4f8e-983e-ddfcbb990178 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:12.298514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 769/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:13.121672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b324e5b-09e0-4a3b-8afc-79dca2c5e28a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:13.122290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b324e5b-09e0-4a3b-8afc-79dca2c5e28a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:13.122290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b324e5b-09e0-4a3b-8afc-79dca2c5e28a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:13.123628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 07:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:13.129937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b523fff-b94b-402a-847e-08700b51d4dc None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:13.130411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b523fff-b94b-402a-847e-08700b51d4dc None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:13.130507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b523fff-b94b-402a-847e-08700b51d4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:13.157565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8b523fff-b94b-402a-847e-08700b51d4dc None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:13.158761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8b523fff-b94b-402a-847e-08700b51d4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:13.166936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 07:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:13.323386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b324e5b-09e0-4a3b-8afc-79dca2c5e28a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:13.346958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b324e5b-09e0-4a3b-8afc-79dca2c5e28a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:13.353011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe19373-61a2-4be8-8eee-822f0bd9304f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:13.353714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19373-61a2-4be8-8eee-822f0bd9304f service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:13.353996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19373-61a2-4be8-8eee-822f0bd9304f service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:13.354260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19373-61a2-4be8-8eee-822f0bd9304f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:13.363082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe19373-61a2-4be8-8eee-822f0bd9304f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:13.364700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe19373-61a2-4be8-8eee-822f0bd9304f service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:13.374550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 757/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:17.312334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c221-7cc8-44e8-9934-d6212e9fadc0 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:17.312910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c221-7cc8-44e8-9934-d6212e9fadc0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:17.312910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c221-7cc8-44e8-9934-d6212e9fadc0 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:17.314246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:17.319690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcbf61-51cc-4e52-9ac7-a0512646d42c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:17.320143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcbf61-51cc-4e52-9ac7-a0512646d42c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:17.320260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcbf61-51cc-4e52-9ac7-a0512646d42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:17.339331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bdfcbf61-51cc-4e52-9ac7-a0512646d42c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:17.340537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bdfcbf61-51cc-4e52-9ac7-a0512646d42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:17.349142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:17.423346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c221-7cc8-44e8-9934-d6212e9fadc0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:17.425477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98e1c221-7cc8-44e8-9934-d6212e9fadc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:17.512920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c221-7cc8-44e8-9934-d6212e9fadc0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:17.517706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905a44a-92cc-43b0-bd7b-b3ecbbad0335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:17.518413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905a44a-92cc-43b0-bd7b-b3ecbbad0335 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:17.518696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905a44a-92cc-43b0-bd7b-b3ecbbad0335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:17.518973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905a44a-92cc-43b0-bd7b-b3ecbbad0335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:17.530491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905a44a-92cc-43b0-bd7b-b3ecbbad0335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:17.531961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905a44a-92cc-43b0-bd7b-b3ecbbad0335 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:17.542727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 772/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:19.619127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae7bb5-d150-433a-9089-602af40b3384 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:19.619748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae7bb5-d150-433a-9089-602af40b3384 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:19.619748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae7bb5-d150-433a-9089-602af40b3384 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:19.621314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:19.628191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98bcb1-2866-41e1-8a06-d15a590c072c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:19.628683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98bcb1-2866-41e1-8a06-d15a590c072c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:19.628786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98bcb1-2866-41e1-8a06-d15a590c072c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:19.648598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cb98bcb1-2866-41e1-8a06-d15a590c072c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:19.656727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cb98bcb1-2866-41e1-8a06-d15a590c072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:19.674026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:19.710897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae7bb5-d150-433a-9089-602af40b3384 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:19.717233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=75 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:11:19.724312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae7bb5-d150-433a-9089-602af40b3384 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:19.728162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4ceba-5b03-4200-b26a-4b4d62f616a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:19.728822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ceba-5b03-4200-b26a-4b4d62f616a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:19.729135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ceba-5b03-4200-b26a-4b4d62f616a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:19.729459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ceba-5b03-4200-b26a-4b4d62f616a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:19.741395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4ceba-5b03-4200-b26a-4b4d62f616a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:19.742841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4ceba-5b03-4200-b26a-4b4d62f616a5 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:19.753400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 760/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:23.156198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb6ac-8f5d-48cc-a817-8b1f1f3eca0d None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:23.157131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb6ac-8f5d-48cc-a817-8b1f1f3eca0d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:23.157131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb6ac-8f5d-48cc-a817-8b1f1f3eca0d None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:23.158671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:11:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:23.164209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ec2b9-5af5-43c3-974a-de7caacdf5c4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:23.164642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ec2b9-5af5-43c3-974a-de7caacdf5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:23.164773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ec2b9-5af5-43c3-974a-de7caacdf5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:23.188639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-431ec2b9-5af5-43c3-974a-de7caacdf5c4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:23.189850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-431ec2b9-5af5-43c3-974a-de7caacdf5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:23.198841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:23.502871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebb6ac-8f5d-48cc-a817-8b1f1f3eca0d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:23.505880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-16ebb6ac-8f5d-48cc-a817-8b1f1f3eca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:23.585611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:11:23.601779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebb6ac-8f5d-48cc-a817-8b1f1f3eca0d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:23.605918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d90c90-0f57-4345-854d-5fe7cb2a97e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:23.606690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d90c90-0f57-4345-854d-5fe7cb2a97e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:23.606968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d90c90-0f57-4345-854d-5fe7cb2a97e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:23.607224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d90c90-0f57-4345-854d-5fe7cb2a97e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:23.616777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d90c90-0f57-4345-854d-5fe7cb2a97e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:23.618316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d90c90-0f57-4345-854d-5fe7cb2a97e9 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:23.628101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 775/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:25.687743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c290a10-82ca-4905-bd8f-df397890af4a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:25.688314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c290a10-82ca-4905-bd8f-df397890af4a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:25.688314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c290a10-82ca-4905-bd8f-df397890af4a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:25.689828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 07:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:11:25.696073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afced1-07b9-4831-8d77-df3be472332e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:25.696539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afced1-07b9-4831-8d77-df3be472332e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:25.696641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afced1-07b9-4831-8d77-df3be472332e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:25.719585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-63afced1-07b9-4831-8d77-df3be472332e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:11:25.721259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-63afced1-07b9-4831-8d77-df3be472332e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:25.730763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 07:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:11:26.033635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c290a10-82ca-4905-bd8f-df397890af4a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:11:26.035386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3c290a10-82ca-4905-bd8f-df397890af4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:11:26.048507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c290a10-82ca-4905-bd8f-df397890af4a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:11:26.052369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3254dc-0ec6-4492-8bb0-d56dc567f519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:11:26.053004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3254dc-0ec6-4492-8bb0-d56dc567f519 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:11:26.053321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3254dc-0ec6-4492-8bb0-d56dc567f519 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:11:26.053599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3254dc-0ec6-4492-8bb0-d56dc567f519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:11:26.062834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3254dc-0ec6-4492-8bb0-d56dc567f519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:11:26.064416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3254dc-0ec6-4492-8bb0-d56dc567f519 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:11:26.075061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 763/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:11:35.714425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=11 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:11:36.070337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=14 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:53.851028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670445e-4ac5-4159-9eec-6cadc2041b4a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:53.852191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670445e-4ac5-4159-9eec-6cadc2041b4a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:53.852702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670445e-4ac5-4159-9eec-6cadc2041b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:53.879752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e670445e-4ac5-4159-9eec-6cadc2041b4a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:53.881674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e670445e-4ac5-4159-9eec-6cadc2041b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:53.891992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 777/1540] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:53.901342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5206-cbfe-4f88-ad61-d2ea6103e834 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:53.901949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5206-cbfe-4f88-ad61-d2ea6103e834 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:53.901949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5206-cbfe-4f88-ad61-d2ea6103e834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:53.931367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c82e5206-cbfe-4f88-ad61-d2ea6103e834 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:53.932618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c82e5206-cbfe-4f88-ad61-d2ea6103e834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:53.944309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 764/1541] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:53.949723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670445e-4ac5-4159-9eec-6cadc2041b4a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:53.967023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670445e-4ac5-4159-9eec-6cadc2041b4a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:53.973700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:53.974827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:53.975324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:53.975809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:53.976777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:53.977339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:53.979249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.012717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 778/1542] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:12:54.014192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42a634-fe51-4a6b-b623-231ff5a0fba1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.014477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42a634-fe51-4a6b-b623-231ff5a0fba1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.014913 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42a634-fe51-4a6b-b623-231ff5a0fba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.031493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.048170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5b42a634-fe51-4a6b-b623-231ff5a0fba1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.052938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95072b4-121b-4b9f-8473-8f998dfdc942 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.053789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5b42a634-fe51-4a6b-b623-231ff5a0fba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.060928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.061690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.062101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.062473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.063329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.064466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:12:54.064466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:12:54.064466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:12:54.068879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.078925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 765/1543] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.088170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fef8a7-8d49-44c5-9fc5-6b21537f788d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.088384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fef8a7-8d49-44c5-9fc5-6b21537f788d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.088649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fef8a7-8d49-44c5-9fc5-6b21537f788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.126564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-07fef8a7-8d49-44c5-9fc5-6b21537f788d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.127721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-07fef8a7-8d49-44c5-9fc5-6b21537f788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.137584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 779/1544] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:12:54.144133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.147840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 766/1545] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:12:54.151838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fef8a7-8d49-44c5-9fc5-6b21537f788d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.170724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fef8a7-8d49-44c5-9fc5-6b21537f788d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.258844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.259536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.259815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.260097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.260826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:54.261149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.262179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:12:54.262179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 12 07:12:54.262179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: Jul 12 07:12:54.263068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.277138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 767/1546] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:12:54.284724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.292313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cce6d-16d3-415f-94a0-cb91eafdb5e0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.296411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325474-5a83-4b7e-bcd6-9ee951fbc7e5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.301017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.301797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.302083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.302360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.303077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.305437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.309341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.310028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.310306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.310579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.311280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.313224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.351238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 780/1547] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:12:54.363076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 768/1548] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/projects => generated 389 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:12:54.370386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.371817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.384110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438d62a-4cbe-4e97-ab85-8b1b676de1e8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.454756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b053b-1843-430a-a52b-5b282140b87a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.460329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2d8c3-3df4-4e07-b57e-0cc73703b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.460488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c6ef1-afaa-415b-95c9-c69666721f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.460616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2d8c3-3df4-4e07-b57e-0cc73703b4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.460720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2d8c3-3df4-4e07-b57e-0cc73703b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.460978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2d8c3-3df4-4e07-b57e-0cc73703b4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.461433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c6ef1-afaa-415b-95c9-c69666721f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.461702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2d8c3-3df4-4e07-b57e-0cc73703b4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.461808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c6ef1-afaa-415b-95c9-c69666721f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.462022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c6ef1-afaa-415b-95c9-c69666721f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.462764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c6ef1-afaa-415b-95c9-c69666721f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.463646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2d8c3-3df4-4e07-b57e-0cc73703b4a2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.464428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c6ef1-afaa-415b-95c9-c69666721f0c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.476582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 769/1549] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:54.477994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf31a64-af83-4474-b309-f3a8bea04f79 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.478325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf31a64-af83-4474-b309-f3a8bea04f79 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.478605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf31a64-af83-4474-b309-f3a8bea04f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.493176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 781/1550] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:54.495106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbbf50-c4bc-4645-8a3d-b41aba9a677c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.495399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbbf50-c4bc-4645-8a3d-b41aba9a677c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.495675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbbf50-c4bc-4645-8a3d-b41aba9a677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.502180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1bf31a64-af83-4474-b309-f3a8bea04f79 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.503386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1bf31a64-af83-4474-b309-f3a8bea04f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.512081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 770/1551] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.513203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacc30e-bbc3-4282-bf87-639b0379b904 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.513488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacc30e-bbc3-4282-bf87-639b0379b904 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.513759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacc30e-bbc3-4282-bf87-639b0379b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.522516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c3bbbf50-c4bc-4645-8a3d-b41aba9a677c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.523983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c3bbbf50-c4bc-4645-8a3d-b41aba9a677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.532339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6aacc30e-bbc3-4282-bf87-639b0379b904 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.533546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6aacc30e-bbc3-4282-bf87-639b0379b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.534169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 782/1552] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.535919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb665406-5b4d-49ce-afcd-e8b98c89c703 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.536462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb665406-5b4d-49ce-afcd-e8b98c89c703 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.536985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb665406-5b4d-49ce-afcd-e8b98c89c703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.542105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 771/1553] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.542901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacc30e-bbc3-4282-bf87-639b0379b904 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.553919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacc30e-bbc3-4282-bf87-639b0379b904 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.557414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-eb665406-5b4d-49ce-afcd-e8b98c89c703 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.558531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.558675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-eb665406-5b4d-49ce-afcd-e8b98c89c703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.559206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.559485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.559763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.560514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.562182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.566486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 783/1554] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.570361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9a4df-abf9-442d-8665-83d498fbae16 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.570654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9a4df-abf9-442d-8665-83d498fbae16 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.570933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9a4df-abf9-442d-8665-83d498fbae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.576047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 772/1555] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:54.576047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.585204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84cb94-6158-41f1-ba78-3bd4a8a87937 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.590638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-03a9a4df-abf9-442d-8665-83d498fbae16 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.590876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e262e-3347-4d23-a8ba-30e18f04d13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.591360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-03a9a4df-abf9-442d-8665-83d498fbae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.591516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e262e-3347-4d23-a8ba-30e18f04d13d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.591800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e262e-3347-4d23-a8ba-30e18f04d13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.592003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e262e-3347-4d23-a8ba-30e18f04d13d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.592735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e262e-3347-4d23-a8ba-30e18f04d13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.594381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e262e-3347-4d23-a8ba-30e18f04d13d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.602655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 784/1556] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.603709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941b6f-3c66-4cda-a27d-a67b735fd7d8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.603992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941b6f-3c66-4cda-a27d-a67b735fd7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.604266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941b6f-3c66-4cda-a27d-a67b735fd7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.608932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 773/1557] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:54.612000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f422e-badf-4637-b207-dc37cb3ccb34 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.612000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f422e-badf-4637-b207-dc37cb3ccb34 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.612000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f422e-badf-4637-b207-dc37cb3ccb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.625963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-42941b6f-3c66-4cda-a27d-a67b735fd7d8 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.627272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-42941b6f-3c66-4cda-a27d-a67b735fd7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.630221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-be0f422e-badf-4637-b207-dc37cb3ccb34 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.631364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-be0f422e-badf-4637-b207-dc37cb3ccb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.635530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 785/1558] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.636616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c797d3d-27b1-422d-8cb8-87664cb98c34 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.636900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c797d3d-27b1-422d-8cb8-87664cb98c34 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.637210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c797d3d-27b1-422d-8cb8-87664cb98c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.642444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 774/1559] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.643625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:54.643796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb7a8-2189-4fc1-9c2c-f9c5ecc5a072 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.644170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb7a8-2189-4fc1-9c2c-f9c5ecc5a072 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.644514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb7a8-2189-4fc1-9c2c-f9c5ecc5a072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.656645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7c797d3d-27b1-422d-8cb8-87664cb98c34 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.657745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:54.657877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7c797d3d-27b1-422d-8cb8-87664cb98c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.665215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e0bdb7a8-2189-4fc1-9c2c-f9c5ecc5a072 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.666337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 786/1560] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.666481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e0bdb7a8-2189-4fc1-9c2c-f9c5ecc5a072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.667568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e99c4-69ed-4897-990d-471857078fac None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.667854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e99c4-69ed-4897-990d-471857078fac None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.668192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e99c4-69ed-4897-990d-471857078fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.677095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 775/1561] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.677095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdb7a8-2189-4fc1-9c2c-f9c5ecc5a072 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.686583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdb7a8-2189-4fc1-9c2c-f9c5ecc5a072 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.689706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-707e99c4-69ed-4897-990d-471857078fac None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.690895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-707e99c4-69ed-4897-990d-471857078fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.691718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.692656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.692936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.693244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] PATH_INFO: `/v3/projects/05ae1ff088b142f3ba6662bb98ad3192/users/666ddde6f8e14ab8b1ac3a2889aa465e/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.698973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 787/1562] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.700584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0ecea-7d4d-4a16-a1b4-85ce61519a52 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.700869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0ecea-7d4d-4a16-a1b4-85ce61519a52 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.701177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0ecea-7d4d-4a16-a1b4-85ce61519a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.710968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ae1ff088b142f3ba6662bb98ad3192, user_id=666ddde6f8e14ab8b1ac3a2889aa465e, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.713679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.719153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9be0ecea-7d4d-4a16-a1b4-85ce61519a52 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.720360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9be0ecea-7d4d-4a16-a1b4-85ce61519a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.727258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 776/1563] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:54 2022] PUT /identity/v3/projects/05ae1ff088b142f3ba6662bb98ad3192/users/666ddde6f8e14ab8b1ac3a2889aa465e/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:54.728058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.779672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 788/1564] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.780523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0ecea-7d4d-4a16-a1b4-85ce61519a52 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.790316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0ecea-7d4d-4a16-a1b4-85ce61519a52 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.794810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.795463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.795866 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.796152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.796893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:54.797228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.799258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.801342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bcd37-3c99-4308-ae5b-aa63d6557b50 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.805857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564d9fe-65d2-4905-b4a0-43c2ce2f8843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.806565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564d9fe-65d2-4905-b4a0-43c2ce2f8843 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.806844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564d9fe-65d2-4905-b4a0-43c2ce2f8843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.807119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564d9fe-65d2-4905-b4a0-43c2ce2f8843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.807874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564d9fe-65d2-4905-b4a0-43c2ce2f8843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.809507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564d9fe-65d2-4905-b4a0-43c2ce2f8843 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.811040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 789/1565] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:12:54.811836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.821591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512edbe-884d-40e8-acf9-23ae89c7e5ce admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.826336 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.827086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.827209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 777/1566] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:54.827537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.827954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.828400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9790c-37ac-47fc-8b20-9f24727e3be6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.828688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9790c-37ac-47fc-8b20-9f24727e3be6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.828955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:54.829121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9790c-37ac-47fc-8b20-9f24727e3be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.829453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.831321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.844557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 790/1567] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:12:54.845981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.848387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-50d9790c-37ac-47fc-8b20-9f24727e3be6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.849396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-50d9790c-37ac-47fc-8b20-9f24727e3be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.856214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20b74-7db9-4a4f-b04e-50bc9633589f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.857496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 778/1568] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.858273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9790c-37ac-47fc-8b20-9f24727e3be6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.864246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940803-9467-4afa-840d-305523127557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.864993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940803-9467-4afa-840d-305523127557 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.865313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940803-9467-4afa-840d-305523127557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.865587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940803-9467-4afa-840d-305523127557 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.866307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940803-9467-4afa-840d-305523127557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:54.866718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940803-9467-4afa-840d-305523127557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.867448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9790c-37ac-47fc-8b20-9f24727e3be6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.868321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940803-9467-4afa-840d-305523127557 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.872124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.872757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.873039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.873337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.874063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:54.874365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.875977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.883661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 791/1569] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:12:54.883661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9115c31-c974-4e4e-9308-3793ff17e6e7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.883661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9115c31-c974-4e4e-9308-3793ff17e6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.883661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9115c31-c974-4e4e-9308-3793ff17e6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.889634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 779/1570] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:12:54.890784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.900546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e1313-30f3-4746-b060-34f68f463333 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.905107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.905891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.906200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.906475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.907223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:54.907534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.909141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.918456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c9115c31-c974-4e4e-9308-3793ff17e6e7 None None] MFA Rules not processed for user `666ddde6f8e14ab8b1ac3a2889aa465e`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:54.919667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c9115c31-c974-4e4e-9308-3793ff17e6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:54.921686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 780/1571] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:12:54.922877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.932890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93ca92-5ad7-4ac6-9e8e-77aae8151e73 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:54.937454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:54.938119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:54.938421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:54.938693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:54.939373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:54.941429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:54.974640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 781/1572] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:54.975528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:54.998678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 792/1573] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:54.999492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9115c31-c974-4e4e-9308-3793ff17e6e7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.010255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9115c31-c974-4e4e-9308-3793ff17e6e7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.044915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abbc1f-c226-4356-afad-a54c80ce7b8b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.048396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:55.050262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.051229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.051501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.051774 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] PATH_INFO: `/v3/projects/2ed408b23e4c48a5861e542d02e0ee64/users/53278c1c534344978ca40d7e2313c02c/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.066270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed408b23e4c48a5861e542d02e0ee64, user_id=53278c1c534344978ca40d7e2313c02c, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.067627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.068314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.068589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.069065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.069185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.069602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.071494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.078006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:55.083408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 782/1574] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:54 2022] PUT /identity/v3/projects/2ed408b23e4c48a5861e542d02e0ee64/users/53278c1c534344978ca40d7e2313c02c/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:55.084350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.099934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 793/1575] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:54 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.100738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.147751 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25705352-92f2-4818-9f0c-2269e0dded46 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.163740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dde143-7958-4d6a-9d79-3673c3ca6500 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.168553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.169257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.169871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.170158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.170852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.173173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.203112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 794/1576] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.205940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.206842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.207843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.208480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.208765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.209654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:12:55.210020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.211923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.224118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 783/1577] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:12:55.224922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.236690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca8695-0db5-4ddf-8495-340d5ea99385 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.272151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba6365-13e0-49fc-be58-921be8b8a1cc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.277590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.278299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.278581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.278854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.279549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.281525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.297870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.298535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.298815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.299188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.299988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.302582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.311281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 795/1578] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.312157 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.337809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 784/1579] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.337809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.352500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:55.368288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:55.376669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef18c2-19ac-472f-bdd4-0a465c35816b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.404969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d28b6c-2f16-4682-9186-fae4c77e9eaf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.409495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0454302-c463-4b3f-a68f-7524c13ea6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.410159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0454302-c463-4b3f-a68f-7524c13ea6db admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.410434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0454302-c463-4b3f-a68f-7524c13ea6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.410974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0454302-c463-4b3f-a68f-7524c13ea6db admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.411675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0454302-c463-4b3f-a68f-7524c13ea6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.413607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0454302-c463-4b3f-a68f-7524c13ea6db admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.436084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.436768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.437058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.437338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.438027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.439291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 785/1580] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.439940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.440627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae192-fd2f-4b2b-b5f3-a57529e1e394 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.440915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae192-fd2f-4b2b-b5f3-a57529e1e394 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.441226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae192-fd2f-4b2b-b5f3-a57529e1e394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.460035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e53ae192-fd2f-4b2b-b5f3-a57529e1e394 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:55.461394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e53ae192-fd2f-4b2b-b5f3-a57529e1e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:55.464818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 796/1581] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.464818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.475183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78f992-ea4c-4848-b3d5-c3c4a98fb405 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.523671 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 786/1582] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:55.524653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ae192-fd2f-4b2b-b5f3-a57529e1e394 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.534028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.534195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ae192-fd2f-4b2b-b5f3-a57529e1e394 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.534809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.535184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.535518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.536357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.538353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.539124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.539786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.540065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.540338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.541009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.542056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:12:55.542056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 12 07:12:55.542056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: Jul 12 07:12:55.543236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.562659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 797/1583] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.562659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.566019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 787/1584] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/users => generated 297 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.566019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.576560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f725e2d-9776-465f-9f40-508fb1fbefbe admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.577237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48e35d-7ce9-4846-b226-b46e5c9bcf9b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.581189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.581772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.581928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.582116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.582466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.582986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.583168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.583277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.583540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.584248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.585121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.586219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.608553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 788/1585] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.609394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.616437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 798/1586] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.616437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.620089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a371a0-23e0-44f5-8540-99c7aae30aeb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.656281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:55.677802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.678027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cd0d8-79d0-4a84-b4aa-44c366e9cb6b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.678490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.678922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.679199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.679891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.681880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.682774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.683474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.683752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.684025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.684706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.686636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.703421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 789/1587] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.704360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.708629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 799/1588] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.708629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.710650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:55.714428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d82760-78e1-4b1a-b41c-48b06afc7591 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.718968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.719706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.719986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.720263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.720988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.722613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.723929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:55.736157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 790/1589] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.736965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.746882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7c2df-4fcd-4d03-aa6d-6bb51059a2c7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.751383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.752033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.752321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.752593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.753331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.754944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.764775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 791/1590] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.765593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.774119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408552b6-b3d1-4a54-bfeb-a7500904955f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.775367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a534a0-dad9-4bdd-b3c5-7862451ade4f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.778248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.778884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.779164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.779463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.780236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.780877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.781181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.781456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.782164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.783739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.788767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.790293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.793802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 792/1591] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.794606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.801626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 800/1592] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:55.802252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.805918 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca6ae0-b12d-47d4-8a14-56f6e1e97bd8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.811860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.812635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb137e0e-1920-46a4-afcf-93499c9543a2 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.812753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.813127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.813492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.814330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.816035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.817880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.818816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.819019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.819294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.820053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.822061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.828396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 793/1593] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.829457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.840298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1cb9c-1955-4cc1-8e4e-b969f3bb99c5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.844988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.845984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.845984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.846231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.846954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.850965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 801/1594] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:12:55 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:12:55.851130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.851242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.860599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 794/1595] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.861453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000ce0-d115-43ba-9c7e-1323da15e86d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.861963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.866833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.867504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.867788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.868090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.869025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.870480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.873601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ecf0f-d5a1-4c3a-9e57-2dd04c50cfc6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.878198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.878932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.879324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.879693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.880486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.885217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 802/1596] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.885383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.885484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.895445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719333c5-d8cf-4912-a8ab-38eca3806815 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.899265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 795/1597] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.900185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.902631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.903436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.904955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.905407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.906523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.908496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.910857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f1bf4-ae7a-4452-a793-4b2cc47b6ebc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.917677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.918533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.918800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.919078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.921883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 803/1598] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.921883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.930047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.933969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.937196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04761865-f514-48b4-9bd3-fdf9b7877c65 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.945665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.946380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.946856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.947055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.947892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.952146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 796/1599] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.952305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.952391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.960814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98c908-b7b5-42c2-baad-643a10f95862 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.963006 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 804/1600] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.963709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.965649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.966294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.966575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.966849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.967573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.969252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:55.975604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25eda6-900d-4adf-9051-0df27c71ace8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:55.983005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 797/1601] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:55.983168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:55.983270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:55.983371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:55.983371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:55.983371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:55.983371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:55.984412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.003548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c081-8b7d-4456-a713-12b0533ef4d1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.004459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 805/1602] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:56.005723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.015301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.017111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.017482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.017766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] PATH_INFO: `/v3/projects/4bfa992c90954fe1995a84182bd63c13/users/f72b131fc86b485b96bf3fdda162246a/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.019691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099670d-decb-4d7c-be7b-db12521015f8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.024623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.025551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.025836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.026116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] PATH_INFO: `/v3/projects/cbae5832fe834b43a2094ab3e803d9ac/users/e857c369a2294c4db5f06a821f34fbf3/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.035536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfa992c90954fe1995a84182bd63c13, user_id=f72b131fc86b485b96bf3fdda162246a, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.038792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.041208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbae5832fe834b43a2094ab3e803d9ac, user_id=e857c369a2294c4db5f06a821f34fbf3, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.042260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:12:56.042260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:12:56.042260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:12:56.044245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.056286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 798/1603] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:55 2022] PUT /identity/v3/projects/4bfa992c90954fe1995a84182bd63c13/users/f72b131fc86b485b96bf3fdda162246a/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.057198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.075120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 806/1604] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:56 2022] PUT /identity/v3/projects/cbae5832fe834b43a2094ab3e803d9ac/users/e857c369a2294c4db5f06a821f34fbf3/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.075120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.118116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:56.138497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ab3d9-39f1-4188-9182-f86d316c9086 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.138748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcacfb-ba4e-41ad-a8bb-fa7dbc2aa67f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.143455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.144349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.144474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.144623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.144966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] PATH_INFO: `/v3/projects/6be635735d174893be20f99110cacbfa/users/583143af14a645238262214515ee148f/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.145960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.146471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.146849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] PATH_INFO: `/v3/projects/02510f7812444c59bec582ff1127aa13/users/7093cc7cf9d1472db33b31e5f98a5141/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.163452 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be635735d174893be20f99110cacbfa, user_id=583143af14a645238262214515ee148f, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.166619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.169278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02510f7812444c59bec582ff1127aa13, user_id=7093cc7cf9d1472db33b31e5f98a5141, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.174191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.193098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 799/1605] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:56 2022] PUT /identity/v3/projects/6be635735d174893be20f99110cacbfa/users/583143af14a645238262214515ee148f/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.196094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.210515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 807/1606] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:56 2022] PUT /identity/v3/projects/02510f7812444c59bec582ff1127aa13/users/7093cc7cf9d1472db33b31e5f98a5141/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.212518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.247991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:56.283670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819a3be-029f-4f78-835b-4bef7c79b5e3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.309770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513124d-b602-45d4-bb11-9bfb56f676a0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.314754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.315558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.315911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.316253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.317783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.319505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.335142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 808/1607] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:56.335142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.345439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ee90d-ee58-44d9-8853-8e8e7c6b9f1e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.351748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce307c5-6753-4406-a0c7-7d644f071a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.352006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.352906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce307c5-6753-4406-a0c7-7d644f071a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.353030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.353218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.353478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce307c5-6753-4406-a0c7-7d644f071a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.353601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] PATH_INFO: `/v3/projects/107fb0f076cc4fc5b61524026f64cb4b/users/3d50ac04cb5f4e74900afdf0ae1171d3/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.353930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce307c5-6753-4406-a0c7-7d644f071a11 admin admin] PATH_INFO: `/v3/projects/c1156aa267ff456b849a9fd0c1a43357/users/8f1fe433179944b2b234ae9f164e8133/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.370589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce307c5-6753-4406-a0c7-7d644f071a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1156aa267ff456b849a9fd0c1a43357, user_id=8f1fe433179944b2b234ae9f164e8133, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.370846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=107fb0f076cc4fc5b61524026f64cb4b, user_id=3d50ac04cb5f4e74900afdf0ae1171d3, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.373274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.373657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce307c5-6753-4406-a0c7-7d644f071a11 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.388220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 800/1608] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:56 2022] PUT /identity/v3/projects/107fb0f076cc4fc5b61524026f64cb4b/users/3d50ac04cb5f4e74900afdf0ae1171d3/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.393092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 809/1609] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:56 2022] PUT /identity/v3/projects/c1156aa267ff456b849a9fd0c1a43357/users/8f1fe433179944b2b234ae9f164e8133/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.393092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb03741-55db-43de-bb75-16dffe11789f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.393092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb03741-55db-43de-bb75-16dffe11789f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.393092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb03741-55db-43de-bb75-16dffe11789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.393513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.421714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8bb03741-55db-43de-bb75-16dffe11789f None None] MFA Rules not processed for user `e857c369a2294c4db5f06a821f34fbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:56.423504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8bb03741-55db-43de-bb75-16dffe11789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:56.468920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335f206-f0e4-4b68-8d67-3a4b1b2a4ec9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.505207 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 810/1610] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:56.506813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393f2a1-0b25-4402-9d3b-d6e1f9f504be None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.507217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393f2a1-0b25-4402-9d3b-d6e1f9f504be None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.507596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393f2a1-0b25-4402-9d3b-d6e1f9f504be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.529516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8393f2a1-0b25-4402-9d3b-d6e1f9f504be None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:56.530935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8393f2a1-0b25-4402-9d3b-d6e1f9f504be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:56.558621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39b211-defb-4f6e-a286-de4e05ba722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.560399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b211-defb-4f6e-a286-de4e05ba722f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.560685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b211-defb-4f6e-a286-de4e05ba722f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.560962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b211-defb-4f6e-a286-de4e05ba722f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.564459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39b211-defb-4f6e-a286-de4e05ba722f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.566088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39b211-defb-4f6e-a286-de4e05ba722f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.578232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 801/1611] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:12:56.579656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfa90-2de8-4ec5-9a22-22ec0e298563 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.580186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfa90-2de8-4ec5-9a22-22ec0e298563 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.580186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfa90-2de8-4ec5-9a22-22ec0e298563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.584002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:56.600005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8f5bfa90-2de8-4ec5-9a22-22ec0e298563 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:56.601308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8f5bfa90-2de8-4ec5-9a22-22ec0e298563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:56.609859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 802/1612] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:56.620055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bfa90-2de8-4ec5-9a22-22ec0e298563 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.627084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:56.631437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bfa90-2de8-4ec5-9a22-22ec0e298563 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.637764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a83ed-71a4-4b50-a578-a40e2c40f469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.640149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a83ed-71a4-4b50-a578-a40e2c40f469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.640510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a83ed-71a4-4b50-a578-a40e2c40f469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.640851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a83ed-71a4-4b50-a578-a40e2c40f469 admin admin] PATH_INFO: `/v3/projects/b52e5f4c43a3469db851b4704f422448/users/ef6395fa14b54c43931ffeaab023d97c/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.644705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 811/1613] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:56.647014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c6bc-a338-448c-bbfb-abbd77d4e77f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.647447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c6bc-a338-448c-bbfb-abbd77d4e77f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.647874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c6bc-a338-448c-bbfb-abbd77d4e77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.661077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a83ed-71a4-4b50-a578-a40e2c40f469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52e5f4c43a3469db851b4704f422448, user_id=ef6395fa14b54c43931ffeaab023d97c, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.664119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a83ed-71a4-4b50-a578-a40e2c40f469 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.671962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e5d4c6bc-a338-448c-bbfb-abbd77d4e77f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:56.674086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e5d4c6bc-a338-448c-bbfb-abbd77d4e77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:56.696071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 803/1614] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:56 2022] PUT /identity/v3/projects/b52e5f4c43a3469db851b4704f422448/users/ef6395fa14b54c43931ffeaab023d97c/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.697545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df5092-ba17-425c-baa6-b08fb189c93f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.697980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df5092-ba17-425c-baa6-b08fb189c93f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.698318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df5092-ba17-425c-baa6-b08fb189c93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.724816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f6df5092-ba17-425c-baa6-b08fb189c93f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:56.727150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f6df5092-ba17-425c-baa6-b08fb189c93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:56.750995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 812/1615] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:56.752615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa5c2d-bb83-47a1-8272-cdea54e2c34b None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.753082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa5c2d-bb83-47a1-8272-cdea54e2c34b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.753620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa5c2d-bb83-47a1-8272-cdea54e2c34b None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.755394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 813/1616] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 07:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:12:56.756122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa5c2d-bb83-47a1-8272-cdea54e2c34b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.772522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa5c2d-bb83-47a1-8272-cdea54e2c34b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.777941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.778999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.779371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.779917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] PATH_INFO: `/v3/projects/4bfa992c90954fe1995a84182bd63c13/users/f72b131fc86b485b96bf3fdda162246a/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.782991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfa992c90954fe1995a84182bd63c13, user_id=f72b131fc86b485b96bf3fdda162246a, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.786243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:56.800430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 804/1617] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:56.800853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 814/1618] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:12:56 2022] PUT /identity/v3/projects/4bfa992c90954fe1995a84182bd63c13/users/f72b131fc86b485b96bf3fdda162246a/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:12:56.802019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.803643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df5092-ba17-425c-baa6-b08fb189c93f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:56.880328 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df5092-ba17-425c-baa6-b08fb189c93f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.895024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2c137-1edb-479f-a238-ebb98738eac9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:56.902693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a8778-28b5-4775-989d-4801a6956f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.904079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8778-28b5-4775-989d-4801a6956f02 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.904696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8778-28b5-4775-989d-4801a6956f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.905035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8778-28b5-4775-989d-4801a6956f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.939143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:56.939851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:56.940137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:56.940418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:56.980645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a8778-28b5-4775-989d-4801a6956f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:56.982351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a8778-28b5-4775-989d-4801a6956f02 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:57.003700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 815/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:57.005390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6842415-ba0e-4024-910b-23e6aa53f4e0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.005630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6842415-ba0e-4024-910b-23e6aa53f4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.006055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6842415-ba0e-4024-910b-23e6aa53f4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.008051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:57.027546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:57.029948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:57.032798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f6842415-ba0e-4024-910b-23e6aa53f4e0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:57.033840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f6842415-ba0e-4024-910b-23e6aa53f4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:57.045861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 816/1620] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:57.047247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 805/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:57.048195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d61b-554e-46e3-9500-f83c074d35d3 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.048681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d61b-554e-46e3-9500-f83c074d35d3 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.049302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:57.049564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d61b-554e-46e3-9500-f83c074d35d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.056440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:57.060334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8a386-8f68-48d7-a562-b2becd9380b6 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:57.064862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc26fff-6609-4221-b3b6-353c83664dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:57.065805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc26fff-6609-4221-b3b6-353c83664dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.066162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc26fff-6609-4221-b3b6-353c83664dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.066525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc26fff-6609-4221-b3b6-353c83664dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.087113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc26fff-6609-4221-b3b6-353c83664dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:57.088675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc26fff-6609-4221-b3b6-353c83664dc9 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:57.105427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 806/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:57.106984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d7606-47a0-4511-990d-dccf7a5bd77f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.107181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d7606-47a0-4511-990d-dccf7a5bd77f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.107469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-41c5d61b-554e-46e3-9500-f83c074d35d3 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:57.107707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d7606-47a0-4511-990d-dccf7a5bd77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.112919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-41c5d61b-554e-46e3-9500-f83c074d35d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:57.153266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-830d7606-47a0-4511-990d-dccf7a5bd77f None None] MFA Rules not processed for user `f72b131fc86b485b96bf3fdda162246a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:57.157921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-830d7606-47a0-4511-990d-dccf7a5bd77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:57.261084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 817/1623] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 07:12:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:57.262358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d61b-554e-46e3-9500-f83c074d35d3 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:57.291074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d61b-554e-46e3-9500-f83c074d35d3 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:57.307261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:57.310290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.310700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.311081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.325444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 807/1624] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:12:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:57.326672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d7606-47a0-4511-990d-dccf7a5bd77f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:57.337034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:57.341546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:57.349830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d7606-47a0-4511-990d-dccf7a5bd77f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:57.359383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:57.360060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.360345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.362342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.366528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 818/1625] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:57.368119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:57.382868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:57.386076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:57.395141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5401f9a-5a43-49b2-827d-0fdadbb13443 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:57.408204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:57.409270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 808/1626] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:57.409884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.410173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.410475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.411301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:57.560382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43474e-51e3-4cf3-aba6-0c68f543c793 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:57.564373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:57.567256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:57.570847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680203bd-236a-40a5-997d-cebe4d4e05f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:57.571532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680203bd-236a-40a5-997d-cebe4d4e05f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.571841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680203bd-236a-40a5-997d-cebe4d4e05f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.572091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680203bd-236a-40a5-997d-cebe4d4e05f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:57.598970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 819/1627] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:57.625551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680203bd-236a-40a5-997d-cebe4d4e05f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:57.627152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680203bd-236a-40a5-997d-cebe4d4e05f7 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:57.658126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 809/1628] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:57.833684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:57.848640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300a569-97e3-4fbf-97dc-f2b1f5af4968 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:57.866296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e7b48-eed1-4989-98f3-cf354de92ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:57.866977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e7b48-eed1-4989-98f3-cf354de92ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:57.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e7b48-eed1-4989-98f3-cf354de92ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:57.867528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e7b48-eed1-4989-98f3-cf354de92ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:58.038365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e7b48-eed1-4989-98f3-cf354de92ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:58.038365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e7b48-eed1-4989-98f3-cf354de92ea9 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:58.063942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 820/1629] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:58.138127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8864019-f48e-4b52-88ff-c7a2af3f12dd None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:58.138787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8864019-f48e-4b52-88ff-c7a2af3f12dd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:58.138787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8864019-f48e-4b52-88ff-c7a2af3f12dd None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:58.140719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 810/1630] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 07:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:12:58.147178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b918fa5-09c5-4144-8725-396ce767b645 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:58.147471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b918fa5-09c5-4144-8725-396ce767b645 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:58.147754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b918fa5-09c5-4144-8725-396ce767b645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:58.160527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:12:58.189343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6b918fa5-09c5-4144-8725-396ce767b645 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:58.190576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6b918fa5-09c5-4144-8725-396ce767b645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:58.210709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 821/1631] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:12:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:12:58.219505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8864019-f48e-4b52-88ff-c7a2af3f12dd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:58.235078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8864019-f48e-4b52-88ff-c7a2af3f12dd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:58.240650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ea2d4-d4aa-470b-af4b-8720295bed5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:58.241401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea2d4-d4aa-470b-af4b-8720295bed5d service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:58.241714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea2d4-d4aa-470b-af4b-8720295bed5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:58.242373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea2d4-d4aa-470b-af4b-8720295bed5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:58.254422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ea2d4-d4aa-470b-af4b-8720295bed5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:12:58.255781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ea2d4-d4aa-470b-af4b-8720295bed5d service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:12:58.274949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 811/1632] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:12:59.757704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ffee3-bd82-4abf-8183-7c9677627e32 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:59.758351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ffee3-bd82-4abf-8183-7c9677627e32 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:59.758351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ffee3-bd82-4abf-8183-7c9677627e32 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:59.760328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 822/1633] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 07:12:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:12:59.774870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99ec1-c282-499a-8424-b2ad948bb1c3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:59.775693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99ec1-c282-499a-8424-b2ad948bb1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:59.775693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99ec1-c282-499a-8424-b2ad948bb1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:59.801722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bca99ec1-c282-499a-8424-b2ad948bb1c3 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:59.805766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bca99ec1-c282-499a-8424-b2ad948bb1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:59.839852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 812/1634] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 07:12:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:59.846923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeac63a-8cca-4f3a-b522-50c4baecf4ea None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:59.847409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeac63a-8cca-4f3a-b522-50c4baecf4ea None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:59.847699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeac63a-8cca-4f3a-b522-50c4baecf4ea None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:59.849205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 823/1635] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 07:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:12:59.854710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3691eb7-cfda-4dca-9b43-3ef4cc1bbabb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:59.855243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3691eb7-cfda-4dca-9b43-3ef4cc1bbabb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:59.855808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3691eb7-cfda-4dca-9b43-3ef4cc1bbabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:59.892402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d3691eb7-cfda-4dca-9b43-3ef4cc1bbabb None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:12:59.896116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d3691eb7-cfda-4dca-9b43-3ef4cc1bbabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:12:59.920654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 813/1636] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:12:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:12:59.936724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeac63a-8cca-4f3a-b522-50c4baecf4ea None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:12:59.957474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeac63a-8cca-4f3a-b522-50c4baecf4ea None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:12:59.971920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:12:59.972640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:12:59.973148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:12:59.973436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:12:59.998725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:00.002224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:00.038984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 824/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:12:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:00.124042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06693181-2865-454f-8490-07df49237a0c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:00.124363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06693181-2865-454f-8490-07df49237a0c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:00.125141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06693181-2865-454f-8490-07df49237a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:00.216178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-06693181-2865-454f-8490-07df49237a0c None None] MFA Rules not processed for user `3d50ac04cb5f4e74900afdf0ae1171d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:00.220981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-06693181-2865-454f-8490-07df49237a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:00.270063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:00.507934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 814/1638] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:00.508704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:00.529064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7dfda-9795-409f-a164-5e6d4d98e008 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:00.551863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:00.554509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:00.556903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:00.557247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:00.558161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:00.561612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:00.653736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 825/1639] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:00 2022] POST /identity/v3/projects => generated 405 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:00.657904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06693181-2865-454f-8490-07df49237a0c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:00.683150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06693181-2865-454f-8490-07df49237a0c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:00.851168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:00.852856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:00.853344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:00.853796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:00.854679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:00.856871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:00.911259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 815/1640] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:00 2022] POST /identity/v3/users => generated 309 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:00.915264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:00.943550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bb182-414a-4921-80a5-1585aa5bb948 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:00.962282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:00.966205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:00.966484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:00.967002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:00.970816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:00.975399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:01.065952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 826/1641] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:01.070344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:01.086023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc169a-142e-410e-879f-dafeeb7e3882 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:01.096591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:01.097720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:01.097998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:01.099812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] PATH_INFO: `/v3/projects/c2e509afc12e45abb45e5c9892b9f241/users/1f09d84f0ef84fb2909f78c0294a1332/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:01.143640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e509afc12e45abb45e5c9892b9f241, user_id=1f09d84f0ef84fb2909f78c0294a1332, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:01.149598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:01.204143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 816/1642] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:01 2022] PUT /identity/v3/projects/c2e509afc12e45abb45e5c9892b9f241/users/1f09d84f0ef84fb2909f78c0294a1332/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:13:01.208193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:01.227220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee78af-04a2-439c-8282-0064e23e1882 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:01.355901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252da28-b473-41d4-8b9e-c13627abb76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:01.356864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252da28-b473-41d4-8b9e-c13627abb76c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:01.357354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252da28-b473-41d4-8b9e-c13627abb76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:01.357788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252da28-b473-41d4-8b9e-c13627abb76c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:01.358937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252da28-b473-41d4-8b9e-c13627abb76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:01.360706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252da28-b473-41d4-8b9e-c13627abb76c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:01.374548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 827/1643] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:01.378377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:01.396003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83da2-4ec6-481a-9d50-2ebf638c71da admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:01.407542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:01.408316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:01.408596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:01.408915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] PATH_INFO: `/v3/projects/c2e509afc12e45abb45e5c9892b9f241/users/1f09d84f0ef84fb2909f78c0294a1332/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:01.411815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e509afc12e45abb45e5c9892b9f241, user_id=1f09d84f0ef84fb2909f78c0294a1332, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:01.414718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:01.445991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 817/1644] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:01 2022] PUT /identity/v3/projects/c2e509afc12e45abb45e5c9892b9f241/users/1f09d84f0ef84fb2909f78c0294a1332/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:13:04.242415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64990-57ee-401e-8b28-948a141ed02f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:04.242415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64990-57ee-401e-8b28-948a141ed02f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:04.243419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64990-57ee-401e-8b28-948a141ed02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:04.288333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4ce64990-57ee-401e-8b28-948a141ed02f None None] MFA Rules not processed for user `53278c1c534344978ca40d7e2313c02c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:04.289649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4ce64990-57ee-401e-8b28-948a141ed02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:04.392509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 828/1645] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:04.393356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:04.484340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:04.496502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598f5c6-eecd-4a93-9454-837e8f873870 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:04.505002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f6fc9-4a3a-4343-9c73-e0548d404aca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:04.505579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f6fc9-4a3a-4343-9c73-e0548d404aca service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:04.505973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f6fc9-4a3a-4343-9c73-e0548d404aca service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:04.506312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f6fc9-4a3a-4343-9c73-e0548d404aca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:04.519901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f6fc9-4a3a-4343-9c73-e0548d404aca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:04.523122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f6fc9-4a3a-4343-9c73-e0548d404aca service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:04.547936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 818/1646] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:04.871081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3ed2-fad2-4411-ae9f-292724174af5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:04.871564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3ed2-fad2-4411-ae9f-292724174af5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:04.871935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3ed2-fad2-4411-ae9f-292724174af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:04.924410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-326e3ed2-fad2-4411-ae9f-292724174af5 None None] MFA Rules not processed for user `1f09d84f0ef84fb2909f78c0294a1332`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:04.926228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-326e3ed2-fad2-4411-ae9f-292724174af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:05.011686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:05.087588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 829/1647] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:07.556050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92a1b9-921a-472c-8c27-7957809aad25 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:07.556050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92a1b9-921a-472c-8c27-7957809aad25 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:07.556711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92a1b9-921a-472c-8c27-7957809aad25 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:07.558141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 819/1648] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:13:07.569933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f1eed-6812-4138-891e-a4c0acbff285 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:07.570519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f1eed-6812-4138-891e-a4c0acbff285 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:07.570690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f1eed-6812-4138-891e-a4c0acbff285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:07.615355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f42f1eed-6812-4138-891e-a4c0acbff285 None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:07.616731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f42f1eed-6812-4138-891e-a4c0acbff285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:07.794337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 830/1649] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:13:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:07.837038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca314ef-808c-4b76-aee7-1f215b05c508 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:07.837337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca314ef-808c-4b76-aee7-1f215b05c508 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:07.837994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca314ef-808c-4b76-aee7-1f215b05c508 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:07.841581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 820/1650] 10.222.0.25 () {54 vars in 960 bytes} [Tue Jul 12 07:13:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:13:07.851168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762186-5321-4965-a288-be4ab4b55a8f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:07.851830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762186-5321-4965-a288-be4ab4b55a8f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:07.852242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762186-5321-4965-a288-be4ab4b55a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:07.904257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-af762186-5321-4965-a288-be4ab4b55a8f None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:07.908468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-af762186-5321-4965-a288-be4ab4b55a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:07.935115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 831/1651] 10.222.0.25 () {58 vars in 1057 bytes} [Tue Jul 12 07:13:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:07.945999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca314ef-808c-4b76-aee7-1f215b05c508 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:07.973932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca314ef-808c-4b76-aee7-1f215b05c508 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:07.989988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:07.991184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:07.993829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:07.994476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:08.020447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:08.023662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:08.069704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 821/1652] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:10.603701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2308f-f53e-4d67-9bdc-c2dd74e621d8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:10.604210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2308f-f53e-4d67-9bdc-c2dd74e621d8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:10.604548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2308f-f53e-4d67-9bdc-c2dd74e621d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:10.649124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-aba2308f-f53e-4d67-9bdc-c2dd74e621d8 None None] MFA Rules not processed for user `53278c1c534344978ca40d7e2313c02c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:10.652633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-aba2308f-f53e-4d67-9bdc-c2dd74e621d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:10.672935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 832/1653] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:10.807952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:10.943893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1e68c-7c57-4ca8-932d-7e3a55f7dcfe service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:10.951676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:10.952951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:10.953499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:10.957439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] PATH_INFO: `/v3/users/e857c369a2294c4db5f06a821f34fbf3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:10.959269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e857c369a2294c4db5f06a821f34fbf3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:10.967753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:11.130288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e857c369a2294c4db5f06a821f34fbf3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:11.131426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e857c369a2294c4db5f06a821f34fbf3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:11.179314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e857c369a2294c4db5f06a821f34fbf3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:11.180575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e857c369a2294c4db5f06a821f34fbf3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:11.212248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e857c369a2294c4db5f06a821f34fbf3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:11.283447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 822/1654] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:10 2022] DELETE /identity/v3/users/e857c369a2294c4db5f06a821f34fbf3 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:11.290576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:11.290799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:11.293718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:11.333916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:11.336287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:11.554109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 833/1655] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:11.601496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:11.606645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:11.830950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdda442-8588-4174-a729-ba97359db89e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:11.844166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd167fd-3c87-4882-a9a9-252183281d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:11.845026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd167fd-3c87-4882-a9a9-252183281d04 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:11.845880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd167fd-3c87-4882-a9a9-252183281d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:11.846238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd167fd-3c87-4882-a9a9-252183281d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:11.873616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd167fd-3c87-4882-a9a9-252183281d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:11.878559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd167fd-3c87-4882-a9a9-252183281d04 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:11.896553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:11.913137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 823/1656] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:12.165861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:12.167906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:12.189534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62d6c-ef7f-46c6-8438-28b496e35056 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:12.208927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:12.209910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:12.210195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:12.210474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] PATH_INFO: `/v3/users/3d50ac04cb5f4e74900afdf0ae1171d3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:12.212765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d50ac04cb5f4e74900afdf0ae1171d3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:12.213852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: WARNING py.warnings [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:13:12.213852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: warnings.warn(msg) Jul 12 07:13:12.213852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: Jul 12 07:13:12.215116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:12.342783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:12.366734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d50ac04cb5f4e74900afdf0ae1171d3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:12.366900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d50ac04cb5f4e74900afdf0ae1171d3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:12.367925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d50ac04cb5f4e74900afdf0ae1171d3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:12.406710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d50ac04cb5f4e74900afdf0ae1171d3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:12.420525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-edc74111-e8cc-4cf4-b301-c63d3d19e79c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d50ac04cb5f4e74900afdf0ae1171d3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:12.455480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 834/1657] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:12 2022] DELETE /identity/v3/users/3d50ac04cb5f4e74900afdf0ae1171d3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:12.570112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57a0a0-d087-4032-b2b7-b6093e03adae None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:12.571115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57a0a0-d087-4032-b2b7-b6093e03adae None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:12.571578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57a0a0-d087-4032-b2b7-b6093e03adae None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:12.574132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 824/1658] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 07:13:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:13:12.586194 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:12.586824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:12.587698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:12.706509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] MFA Rules not processed for user `754c137a554941d99f7ef9c380a55ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:12.708984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:12.889658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 835/1659] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:13:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:13.063724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57a0a0-d087-4032-b2b7-b6093e03adae None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:13.066383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8e57a0a0-d087-4032-b2b7-b6093e03adae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.084878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:13.086751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.110232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e28d0-3d4c-40ad-9b29-f582981b0751 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:13.123722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635ceee-6c46-43af-9db3-70434bf18548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:13.124776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ceee-6c46-43af-9db3-70434bf18548 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:13.125296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ceee-6c46-43af-9db3-70434bf18548 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:13.125768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ceee-6c46-43af-9db3-70434bf18548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:13.141553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635ceee-6c46-43af-9db3-70434bf18548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:13.154351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635ceee-6c46-43af-9db3-70434bf18548 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:13.179748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 836/1660] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:13:13.218271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57a0a0-d087-4032-b2b7-b6093e03adae None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:13.226030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:13.227316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:13.228301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:13.228788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] PATH_INFO: `/v3/users/666ddde6f8e14ab8b1ac3a2889aa465e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:13.230337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=666ddde6f8e14ab8b1ac3a2889aa465e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:13.232866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:13.317921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666ddde6f8e14ab8b1ac3a2889aa465e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.318633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666ddde6f8e14ab8b1ac3a2889aa465e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.335800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666ddde6f8e14ab8b1ac3a2889aa465e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.336604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666ddde6f8e14ab8b1ac3a2889aa465e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.347896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666ddde6f8e14ab8b1ac3a2889aa465e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.375256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 825/1661] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:13 2022] DELETE /identity/v3/users/666ddde6f8e14ab8b1ac3a2889aa465e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:13.378002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:13.380205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:13.381204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:13.395706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:13.402817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.432837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:13.434116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.603649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 837/1662] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:13.610719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab7c74-912a-42ba-8636-d0b70e8bff48 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:13.611091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:13.614583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.627192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:13.628152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:13.628740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:13.629150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] PATH_INFO: `/v3/projects/cbae5832fe834b43a2094ab3e803d9ac` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:13.632095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbae5832fe834b43a2094ab3e803d9ac)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:13.634717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 07:13:13.634717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 12 07:13:13.634717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: Jul 12 07:13:13.640804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:13.643890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10c0a3-514f-4a47-bf0a-3af9c075fafa None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:13.656444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:13.657784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:13.659088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:13.659601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] PATH_INFO: `/v3/projects/107fb0f076cc4fc5b61524026f64cb4b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:13.661766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] RBAC: Authorizing `identity:delete_project(project_id=107fb0f076cc4fc5b61524026f64cb4b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:13.667136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:13.693694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:13.760848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:13.761116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbae5832fe834b43a2094ab3e803d9ac', 'request_id': 'req-acedd823-7d6c-4c18-99cc-3c022c6269e8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.776090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107fb0f076cc4fc5b61524026f64cb4b', 'request_id': 'req-cad8add6-35fb-4661-9766-1a24334a536d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.779476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbae5832fe834b43a2094ab3e803d9ac', 'request_id': 'req-acedd823-7d6c-4c18-99cc-3c022c6269e8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.794896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107fb0f076cc4fc5b61524026f64cb4b', 'request_id': 'req-cad8add6-35fb-4661-9766-1a24334a536d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.813422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107fb0f076cc4fc5b61524026f64cb4b', 'request_id': 'req-cad8add6-35fb-4661-9766-1a24334a536d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.825577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbae5832fe834b43a2094ab3e803d9ac', 'request_id': 'req-acedd823-7d6c-4c18-99cc-3c022c6269e8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.836537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] The token cache is being invalidate because project 107fb0f076cc4fc5b61524026f64cb4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:13.836899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.839236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.841740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 838/1663] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:13 2022] DELETE /identity/v3/projects/107fb0f076cc4fc5b61524026f64cb4b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:13.843112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:13.844936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.848773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] The token cache is being invalidate because project cbae5832fe834b43a2094ab3e803d9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:13.849419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.851186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:13.854876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 826/1664] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:13 2022] DELETE /identity/v3/projects/cbae5832fe834b43a2094ab3e803d9ac => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:13.856606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:13.858359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.931901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8add6-35fb-4661-9766-1a24334a536d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:13.941374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8833564-6907-4969-a5c8-159d02507d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:13.942463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833564-6907-4969-a5c8-159d02507d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:13.942941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833564-6907-4969-a5c8-159d02507d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:13.943418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833564-6907-4969-a5c8-159d02507d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:13.945700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8833564-6907-4969-a5c8-159d02507d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:13.976525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedd823-7d6c-4c18-99cc-3c022c6269e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:14.108234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8833564-6907-4969-a5c8-159d02507d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:14.109359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8833564-6907-4969-a5c8-159d02507d0a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:14.138672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:14.139606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:14.139891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:14.140575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] PATH_INFO: `/v3/users/1f09d84f0ef84fb2909f78c0294a1332` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:14.143805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f09d84f0ef84fb2909f78c0294a1332)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:14.148564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 839/1665] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:14.149991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16998d-b53d-4f7b-b93b-345c07872a40 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:14.150279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16998d-b53d-4f7b-b93b-345c07872a40 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:14.150554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16998d-b53d-4f7b-b93b-345c07872a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:14.153375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:14.230697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3b16998d-b53d-4f7b-b93b-345c07872a40 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:14.236365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3b16998d-b53d-4f7b-b93b-345c07872a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:14.269750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 840/1666] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:14.272352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c60b9-2e69-4a50-b0ea-c9fe7e967cd0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:14.272651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c60b9-2e69-4a50-b0ea-c9fe7e967cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:14.273040 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f09d84f0ef84fb2909f78c0294a1332'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:14.274728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f09d84f0ef84fb2909f78c0294a1332'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:14.279089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c60b9-2e69-4a50-b0ea-c9fe7e967cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:14.294222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:14.313886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f09d84f0ef84fb2909f78c0294a1332'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:14.315763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f09d84f0ef84fb2909f78c0294a1332'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:14.333184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-028c60b9-2e69-4a50-b0ea-c9fe7e967cd0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:14.335257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f09d84f0ef84fb2909f78c0294a1332'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:14.335587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-028c60b9-2e69-4a50-b0ea-c9fe7e967cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:14.361037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 827/1667] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:13 2022] DELETE /identity/v3/users/1f09d84f0ef84fb2909f78c0294a1332 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:14.361782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 841/1668] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:14.362703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:14.525270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857f94d-80d8-4432-9605-ce0c4cb88fb6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:14.525564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857f94d-80d8-4432-9605-ce0c4cb88fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:14.525919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857f94d-80d8-4432-9605-ce0c4cb88fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:14.526919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36dafa-fb49-438d-8ea3-6438d44b0035 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:14.536729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:14.538564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:14.580617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:14.581840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:14.588790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:14.590742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:14.596785 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:14.606274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5857f94d-80d8-4432-9605-ce0c4cb88fb6 None None] MFA Rules not processed for user `583143af14a645238262214515ee148f`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:14.613080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5857f94d-80d8-4432-9605-ce0c4cb88fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:14.656004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 828/1669] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:13:14.658430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:14.685849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8f85f-6b5e-4f4c-a21d-da58c12eaa91 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:14.707625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:14.707956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:14.708245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:14.708519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:14.709271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:14.720193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:14.757887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:14.793873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 829/1670] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:14 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:14.794776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:14.798960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:14.829713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 842/1671] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:14.837832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857f94d-80d8-4432-9605-ce0c4cb88fb6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:14.906540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857f94d-80d8-4432-9605-ce0c4cb88fb6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:15.009809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55426-7615-4eb5-b468-e2794a46f8c7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:15.037158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:15.037852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.038136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.038412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.053777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:15.088270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:15.094300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:15.132409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 830/1672] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:15.136216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:15.142672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:15.185600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:15.186974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.187397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.188328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.197956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:15.205097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:15.267665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 843/1673] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:14 2022] POST /identity/v3/users => generated 283 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:15.274329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:15.288902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:15.309611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17777f-fd65-4eb2-bc46-4fd29bab4fcc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:15.327630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:15.328347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.328628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.328901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.329698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:15.340770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:15.380234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 844/1674] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:15.388638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:15.419743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65fffe-0a2a-4585-922c-0f9373b63972 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:15.422011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c33e8-2e67-4606-9ae8-8911cf970cf9 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:15.443956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ea484-5bf3-4e96-bbd1-c4c126a369ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:15.444649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea484-5bf3-4e96-bbd1-c4c126a369ca service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.444934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea484-5bf3-4e96-bbd1-c4c126a369ca service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.445241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea484-5bf3-4e96-bbd1-c4c126a369ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.454788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:15.455990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.456435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.456942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.458004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:15.459930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:15.475281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 845/1675] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:15.479334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:15.495558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f64f62-a45c-4261-916f-18716dbfeea4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:15.502828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3eaf0-d5ec-45f9-a7ab-f625fdc95c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:15.504011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3eaf0-d5ec-45f9-a7ab-f625fdc95c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.504545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3eaf0-d5ec-45f9-a7ab-f625fdc95c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.505916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3eaf0-d5ec-45f9-a7ab-f625fdc95c82 admin admin] PATH_INFO: `/v3/projects/466c236ee03c43b995d9f8c2849f74c9/users/8b28e89a0173461aa5faefc14948c616/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.531326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3eaf0-d5ec-45f9-a7ab-f625fdc95c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=466c236ee03c43b995d9f8c2849f74c9, user_id=8b28e89a0173461aa5faefc14948c616, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:15.536014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3eaf0-d5ec-45f9-a7ab-f625fdc95c82 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:15.558868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ea484-5bf3-4e96-bbd1-c4c126a369ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:15.565239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ea484-5bf3-4e96-bbd1-c4c126a369ca service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:15.567698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 846/1676] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:15 2022] PUT /identity/v3/projects/466c236ee03c43b995d9f8c2849f74c9/users/8b28e89a0173461aa5faefc14948c616/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:15.579282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.579616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.579842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.622977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 831/1677] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:15.641939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:15.647089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:15.648931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:15.688380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] MFA Rules not processed for user `8b28e89a0173461aa5faefc14948c616`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:15.691699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:15.802935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] MFA Rules not processed for user `7093cc7cf9d1472db33b31e5f98a5141`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:15.811294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:15.934091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 847/1678] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:13:15.935922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:15.939579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:16.005321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:16.131922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482bdd-6037-4d59-a646-a792b0a331c5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:16.146295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:16.147266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:16.147544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:16.147818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] PATH_INFO: `/v3/projects/c2e509afc12e45abb45e5c9892b9f241` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:16.149472 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e509afc12e45abb45e5c9892b9f241)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:16.156329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:16.158184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 832/1679] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 535 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:16.159192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:16.184408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:16.330189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e509afc12e45abb45e5c9892b9f241', 'request_id': 'req-e1f2dd21-7a8e-4283-b793-e43d19ea3656'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:16.350818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e509afc12e45abb45e5c9892b9f241', 'request_id': 'req-e1f2dd21-7a8e-4283-b793-e43d19ea3656'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:16.369146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e509afc12e45abb45e5c9892b9f241', 'request_id': 'req-e1f2dd21-7a8e-4283-b793-e43d19ea3656'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:16.401492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] The token cache is being invalidate because project c2e509afc12e45abb45e5c9892b9f241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:16.401492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:16.401787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5ef9f-d2ea-47a3-88ee-21c5979cd323 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:16.401881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e1f2dd21-7a8e-4283-b793-e43d19ea3656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:16.406253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 848/1680] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:15 2022] DELETE /identity/v3/projects/c2e509afc12e45abb45e5c9892b9f241 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:16.407340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644db4c-a9a6-49f2-8c1f-27ca8011660c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:16.408745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644db4c-a9a6-49f2-8c1f-27ca8011660c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:16.408745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644db4c-a9a6-49f2-8c1f-27ca8011660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:16.451244 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:16.469012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b644db4c-a9a6-49f2-8c1f-27ca8011660c None None] MFA Rules not processed for user `7093cc7cf9d1472db33b31e5f98a5141`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:16.471427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b644db4c-a9a6-49f2-8c1f-27ca8011660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:16.599468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e587550-8934-4ce0-b0b4-b8dcaeeb261a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:16.600565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587550-8934-4ce0-b0b4-b8dcaeeb261a service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:16.601103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587550-8934-4ce0-b0b4-b8dcaeeb261a service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:16.601440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587550-8934-4ce0-b0b4-b8dcaeeb261a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:16.610683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5e587550-8934-4ce0-b0b4-b8dcaeeb261a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:16.662998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 849/1681] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:16.666462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:16.670886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:16.672415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:16.753780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] MFA Rules not processed for user `ef6395fa14b54c43931ffeaab023d97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:16.757663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:16.807872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e587550-8934-4ce0-b0b4-b8dcaeeb261a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:16.811914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e587550-8934-4ce0-b0b4-b8dcaeeb261a service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:16.854714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 833/1682] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:16.859937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:16.862186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:16.864295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:16.917164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:16.920684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:16.935813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:16.996415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 850/1683] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:16.997665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:17.000020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:17.160825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 834/1684] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:17.166058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:17.170660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:17.281494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06136c4-036e-46bf-bd2a-0071ff3374f2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:17.306361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:17.307286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:17.307568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:17.307921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] PATH_INFO: `/v3/projects/05ae1ff088b142f3ba6662bb98ad3192` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:17.313987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05ae1ff088b142f3ba6662bb98ad3192)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:17.316502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:17.411460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79f17a-d9ab-4c72-868a-183365306399 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:17.419514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b5f08-1205-431e-b559-aff696d4ce3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:17.422178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b5f08-1205-431e-b559-aff696d4ce3a service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:17.422580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b5f08-1205-431e-b559-aff696d4ce3a service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:17.422959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b5f08-1205-431e-b559-aff696d4ce3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:17.432926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-492b5f08-1205-431e-b559-aff696d4ce3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:17.494663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b5f08-1205-431e-b559-aff696d4ce3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:17.497786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05ae1ff088b142f3ba6662bb98ad3192', 'request_id': 'req-f5cca591-114a-4f09-bb1f-456a5bf7680a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:17.515058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b5f08-1205-431e-b559-aff696d4ce3a service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:17.521179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05ae1ff088b142f3ba6662bb98ad3192', 'request_id': 'req-f5cca591-114a-4f09-bb1f-456a5bf7680a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:17.533459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:17.550342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05ae1ff088b142f3ba6662bb98ad3192', 'request_id': 'req-f5cca591-114a-4f09-bb1f-456a5bf7680a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:17.591645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] The token cache is being invalidate because project 05ae1ff088b142f3ba6662bb98ad3192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:17.591937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:17.592796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:17.596398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 851/1685] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:16 2022] DELETE /identity/v3/projects/05ae1ff088b142f3ba6662bb98ad3192 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:17.600126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:17.606109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 835/1686] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:17.609986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:17.610843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:17.618262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:17.618576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:17.689303 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:17.735223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:17.983208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cca591-114a-4f09-bb1f-456a5bf7680a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:18.004129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:18.004860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:18.005169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:18.005442 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:18.013497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 836/1687] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:18.014438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:18.020856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:18.023590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:18.059485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d16ce-5eaf-41f2-9d02-20ef0fea9e6f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:18.065283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:18.076176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:18.077635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:18.078176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:18.078622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] PATH_INFO: `/v3/users/f72b131fc86b485b96bf3fdda162246a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:18.081660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72b131fc86b485b96bf3fdda162246a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:18.095035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:18.147431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:18.149864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:18.179481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 852/1688] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:18.182307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:18.224170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72b131fc86b485b96bf3fdda162246a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:18.226279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72b131fc86b485b96bf3fdda162246a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:18.226851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560339-348e-4fe4-97b8-99f49d7356a9 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:18.243138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7907e5-97dc-4e8d-8ff6-14238ba0101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:18.243868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7907e5-97dc-4e8d-8ff6-14238ba0101b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:18.244099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7907e5-97dc-4e8d-8ff6-14238ba0101b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:18.244370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7907e5-97dc-4e8d-8ff6-14238ba0101b admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:18.245143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7907e5-97dc-4e8d-8ff6-14238ba0101b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:18.245884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7907e5-97dc-4e8d-8ff6-14238ba0101b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:18.250910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7907e5-97dc-4e8d-8ff6-14238ba0101b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:18.260555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72b131fc86b485b96bf3fdda162246a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:18.264376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72b131fc86b485b96bf3fdda162246a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:18.279604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 853/1689] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:13:18.285126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81eaef-7b3d-44a4-bed0-140b39a48d48 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:18.285368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81eaef-7b3d-44a4-bed0-140b39a48d48 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:18.285651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81eaef-7b3d-44a4-bed0-140b39a48d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:18.287720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72b131fc86b485b96bf3fdda162246a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:18.322375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 837/1690] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:18 2022] DELETE /identity/v3/users/f72b131fc86b485b96bf3fdda162246a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:18.333224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:18.341490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:18.435186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-aa81eaef-7b3d-44a4-bed0-140b39a48d48 None None] MFA Rules not processed for user `8f1fe433179944b2b234ae9f164e8133`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:18.437328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-aa81eaef-7b3d-44a4-bed0-140b39a48d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:18.616617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86225d4-7251-439e-9146-35309ec545a1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:18.636842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:18.637457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:18.637738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:18.638015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:18.638726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:18.647176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:18.706713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 854/1691] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:18.708581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:18.708875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:18.709300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:18.709516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:18.753767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 838/1692] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:18 2022] POST /identity/v3/projects => generated 373 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:18.757467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:18.779781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] MFA Rules not processed for user `8f1fe433179944b2b234ae9f164e8133`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:18.784052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:18.938773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 855/1693] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:18.953406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:18.955863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:18.974600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af52393-b09f-4a5e-912a-8863c99f6d7c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:18.980753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:18.981740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:18.983690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:18.983690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:18.983690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:19.024875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:19.066174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 839/1694] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:18 2022] POST /identity/v3/users => generated 287 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:19.092522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:19.111247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9e47d-3b22-401b-97e5-cf740fef3e02 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:19.118965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:19.120264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.120744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.121778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.122758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:19.125744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:19.139180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:19.151082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 840/1695] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:19.152615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:19.172063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c9e72-6cfe-4686-b8cc-fe2b7a2f2d93 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:19.188578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:19.192096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.193686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.194783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.196504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:19.198252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:19.219969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 841/1696] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:19.225757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:19.233473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ad4f9-0c5d-46c4-ba07-198e72e79348 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:19.241916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee6cdf-117a-4dff-9c50-ec2f5924bedc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:19.242966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6cdf-117a-4dff-9c50-ec2f5924bedc service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.244215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6cdf-117a-4dff-9c50-ec2f5924bedc service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.244799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6cdf-117a-4dff-9c50-ec2f5924bedc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.251466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897ea7f-8a07-4dff-9116-c8d88a27ad28 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:19.259875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fad297-75f9-4d64-bbcc-3be8055a9bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:19.261415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fad297-75f9-4d64-bbcc-3be8055a9bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.262048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fad297-75f9-4d64-bbcc-3be8055a9bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.262152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fad297-75f9-4d64-bbcc-3be8055a9bcd admin admin] PATH_INFO: `/v3/projects/24f3d9d4af2d4f509cce46bde47eb0c2/users/1e8458e26d7d490eb76e01b075c42cf1/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.264009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee6cdf-117a-4dff-9c50-ec2f5924bedc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:19.265881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee6cdf-117a-4dff-9c50-ec2f5924bedc service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:19.281351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 856/1697] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:19.295011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fad297-75f9-4d64-bbcc-3be8055a9bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f3d9d4af2d4f509cce46bde47eb0c2, user_id=1e8458e26d7d490eb76e01b075c42cf1, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:19.302571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fad297-75f9-4d64-bbcc-3be8055a9bcd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:19.337230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 842/1698] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:19 2022] PUT /identity/v3/projects/24f3d9d4af2d4f509cce46bde47eb0c2/users/1e8458e26d7d490eb76e01b075c42cf1/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:19.348695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.349099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.349524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.396087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] MFA Rules not processed for user `1e8458e26d7d490eb76e01b075c42cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:19.401241 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:19.562005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.562296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.563211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.583359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 857/1699] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:19.592447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:19.605282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:19.679457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:19.683459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:19.695781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750c35-bbed-4065-aace-4a3bdd8556a4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:19.699935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:19.700719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.701356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.701768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.724387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:19.727127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:19.754570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 858/1700] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:13:19.778709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:19.833803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 843/1701] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:13:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:19.908440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:19.917013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914137a-5f85-4ece-a194-2bc71bb9249f service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:19.924165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:19.925750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:19.926174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:19.926545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] PATH_INFO: `/v3/users/1e8458e26d7d490eb76e01b075c42cf1` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:19.927922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e8458e26d7d490eb76e01b075c42cf1)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:19.930290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:19.973883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:19.977086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:20.013503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:20.025839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e8458e26d7d490eb76e01b075c42cf1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:20.026581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e8458e26d7d490eb76e01b075c42cf1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:20.027163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e8458e26d7d490eb76e01b075c42cf1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:20.076807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e8458e26d7d490eb76e01b075c42cf1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:20.170595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c23567b6-17c1-4804-a74e-4c45e56e68c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e8458e26d7d490eb76e01b075c42cf1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:20.213748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 859/1702] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:19 2022] DELETE /identity/v3/users/1e8458e26d7d490eb76e01b075c42cf1 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:20.213748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce19480-a07b-43a3-8527-2d156a7ecd50 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:20.213748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce19480-a07b-43a3-8527-2d156a7ecd50 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:20.214071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce19480-a07b-43a3-8527-2d156a7ecd50 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:20.245657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 860/1703] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 07:13:20 2022] GET /identity => generated 270 bytes in 34 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:13:20.248372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7a8fc-7daa-4f4f-a757-b0f7510e40fb None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:20.252258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7a8fc-7daa-4f4f-a757-b0f7510e40fb None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:20.255799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7a8fc-7daa-4f4f-a757-b0f7510e40fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:20.289977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc56a4-376e-41cf-8d6e-14308828e77f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:20.309227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e5e7a8fc-7daa-4f4f-a757-b0f7510e40fb None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:20.311208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e5e7a8fc-7daa-4f4f-a757-b0f7510e40fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:20.515088 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:20.515808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:20.516091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:20.516363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:20.521765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:20.563772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 861/1704] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:20.564568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:20.565269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:20.565557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:20.595144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] MFA Rules not processed for user `754c137a554941d99f7ef9c380a55ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:20.599929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:20.707684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:20.710886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:20.741352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 844/1705] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:13:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:13:20.742373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:20.744880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:20.815363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 862/1706] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:13:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:20.954368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:20.964627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:20.966158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:21.031956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7d3c4-73d5-4f69-8d58-fe700fc260ef None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:21.062330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283be146-909c-4824-82fe-317535f52a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:21.063772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283be146-909c-4824-82fe-317535f52a4f service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:21.063772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283be146-909c-4824-82fe-317535f52a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:21.063772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283be146-909c-4824-82fe-317535f52a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:21.069960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952be64-1717-4f6e-bc6d-8cd805fd25a1 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:21.090474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:21.091183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:21.091465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:21.091740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:21.116223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283be146-909c-4824-82fe-317535f52a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:21.117082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283be146-909c-4824-82fe-317535f52a4f service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:21.139116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:21.153582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:21.160579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 863/1707] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:21.167763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74ce9f-84eb-4b15-b357-1f606387e602 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:21.168120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74ce9f-84eb-4b15-b357-1f606387e602 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:21.168438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74ce9f-84eb-4b15-b357-1f606387e602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:21.206160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 845/1708] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:21.240116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9d74ce9f-84eb-4b15-b357-1f606387e602 None None] MFA Rules not processed for user `583143af14a645238262214515ee148f`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:21.245975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9d74ce9f-84eb-4b15-b357-1f606387e602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:21.274264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:21.575693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 864/1709] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:22.134463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:22.139050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:22.175723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f9a93-dae7-46d7-ac20-01e7dd11e9d9 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:22.209923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:22.212174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:22.214187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:22.215905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] PATH_INFO: `/v3/projects/4bfa992c90954fe1995a84182bd63c13` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:22.219854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bfa992c90954fe1995a84182bd63c13)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:22.228741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:22.255644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc19f0-d16f-42d7-8447-3aac5e2aaa83 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:22.256254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc19f0-d16f-42d7-8447-3aac5e2aaa83 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:22.256254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc19f0-d16f-42d7-8447-3aac5e2aaa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:22.334363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-72dc19f0-d16f-42d7-8447-3aac5e2aaa83 None None] MFA Rules not processed for user `7093cc7cf9d1472db33b31e5f98a5141`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:22.345028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-72dc19f0-d16f-42d7-8447-3aac5e2aaa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:22.579126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bfa992c90954fe1995a84182bd63c13', 'request_id': 'req-2a0c4507-0bdb-4ad9-9dba-226139d1219e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:22.629705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bfa992c90954fe1995a84182bd63c13', 'request_id': 'req-2a0c4507-0bdb-4ad9-9dba-226139d1219e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:22.717221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bfa992c90954fe1995a84182bd63c13', 'request_id': 'req-2a0c4507-0bdb-4ad9-9dba-226139d1219e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:22.834273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] The token cache is being invalidate because project 4bfa992c90954fe1995a84182bd63c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:22.834604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:22.835829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2a0c4507-0bdb-4ad9-9dba-226139d1219e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:22.845301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 846/1710] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:22 2022] DELETE /identity/v3/projects/4bfa992c90954fe1995a84182bd63c13 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:22.847133 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f14c84-c50a-492c-83d3-545815347128 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:22.847419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f14c84-c50a-492c-83d3-545815347128 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:22.847693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f14c84-c50a-492c-83d3-545815347128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:22.852335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 865/1711] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 596 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:22.903427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-67f14c84-c50a-492c-83d3-545815347128 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:22.911059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-67f14c84-c50a-492c-83d3-545815347128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:23.151930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 847/1712] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:23.162521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db2f3-b7c4-42d1-964e-9db7846ea9cd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:23.163078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db2f3-b7c4-42d1-964e-9db7846ea9cd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:23.163574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db2f3-b7c4-42d1-964e-9db7846ea9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:23.247684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cd6db2f3-b7c4-42d1-964e-9db7846ea9cd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:23.252420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cd6db2f3-b7c4-42d1-964e-9db7846ea9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:23.286047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 866/1713] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:23.290650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f14c84-c50a-492c-83d3-545815347128 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:23.310692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f14c84-c50a-492c-83d3-545815347128 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:23.346023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:23.346649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:23.351950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:23.352366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:23.364415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:23.364709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:23.374482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:23.441709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db2f3-b7c4-42d1-964e-9db7846ea9cd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:23.462140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 848/1714] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:13:23.469712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db2f3-b7c4-42d1-964e-9db7846ea9cd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:23.493617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:23.499428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:23.501529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:23.504396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:23.505854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:23.517494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:23.675899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 867/1715] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:23 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:23.677534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:23.694645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86c065-fdd2-47a1-a025-09f32123fc71 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:23.871650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:23.938220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:23.945616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:23.946802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:23.946802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:23.949402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:23.951766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:24.039463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 849/1716] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:23 2022] POST /identity/v3/users => generated 284 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:24.044208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:24.071584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22769aa-6f9b-4ebd-b38d-af3bc2f99585 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:24.159218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:24.163549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:24.166299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:24.174598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:24.184832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:24.201843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:24.282334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 868/1717] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:24.285225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:24.305707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980d11-260c-4e1f-81f4-0c28adca386b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:24.332199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:24.333231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:24.336282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:24.337507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:24.339315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:24.341168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:24.389994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:24.395810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:24.403421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 850/1718] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:24.404548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:24.424257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09758-4e1a-4466-8bdf-abadb6483856 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:24.439778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:24.441137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:24.444305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:24.444786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] PATH_INFO: `/v3/projects/24f3d9d4af2d4f509cce46bde47eb0c2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:24.446436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f3d9d4af2d4f509cce46bde47eb0c2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:24.451973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7d53a-966d-45d7-aa5b-c34837814149 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:24.452466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:24.476196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e22845-0a4a-44b7-af60-57776b5377b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:24.477318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e22845-0a4a-44b7-af60-57776b5377b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:24.477712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e22845-0a4a-44b7-af60-57776b5377b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:24.478081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e22845-0a4a-44b7-af60-57776b5377b9 admin admin] PATH_INFO: `/v3/projects/353c98633dc14ea49f5134d3477223e8/users/c31f96e10614498d9ad21435767d6cde/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:24.548128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e22845-0a4a-44b7-af60-57776b5377b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=353c98633dc14ea49f5134d3477223e8, user_id=c31f96e10614498d9ad21435767d6cde, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:24.565523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e22845-0a4a-44b7-af60-57776b5377b9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:24.646527 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 851/1719] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:24 2022] PUT /identity/v3/projects/353c98633dc14ea49f5134d3477223e8/users/c31f96e10614498d9ad21435767d6cde/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:24.680308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:24.686186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:24.686840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:24.704419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f3d9d4af2d4f509cce46bde47eb0c2', 'request_id': 'req-e786ee09-98ce-43dc-9e41-487a36e00366'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:24.727755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f3d9d4af2d4f509cce46bde47eb0c2', 'request_id': 'req-e786ee09-98ce-43dc-9e41-487a36e00366'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:24.758005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f3d9d4af2d4f509cce46bde47eb0c2', 'request_id': 'req-e786ee09-98ce-43dc-9e41-487a36e00366'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:24.793378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] The token cache is being invalidate because project 24f3d9d4af2d4f509cce46bde47eb0c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:24.798224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:24.802117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e786ee09-98ce-43dc-9e41-487a36e00366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:24.824422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 869/1720] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:24 2022] DELETE /identity/v3/projects/24f3d9d4af2d4f509cce46bde47eb0c2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:24.837031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308dff-296b-41a6-a719-44efa106d654 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:24.837332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] MFA Rules not processed for user `c31f96e10614498d9ad21435767d6cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:24.839122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:24.839868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308dff-296b-41a6-a719-44efa106d654 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:24.840420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308dff-296b-41a6-a719-44efa106d654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:24.864855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:24.906947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f7308dff-296b-41a6-a719-44efa106d654 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:24.915148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f7308dff-296b-41a6-a719-44efa106d654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:25.144936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 852/1721] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:25.149482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:25.151776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:25.214603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 870/1722] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:25.218253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704217fc-f0c8-4f38-8db3-5f450a9e7fe6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:25.218663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704217fc-f0c8-4f38-8db3-5f450a9e7fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:25.219041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704217fc-f0c8-4f38-8db3-5f450a9e7fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:25.304357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-704217fc-f0c8-4f38-8db3-5f450a9e7fe6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:25.316665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-704217fc-f0c8-4f38-8db3-5f450a9e7fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:25.361072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 871/1723] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:25.364478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704217fc-f0c8-4f38-8db3-5f450a9e7fe6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:25.404080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704217fc-f0c8-4f38-8db3-5f450a9e7fe6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:25.425435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:25.426300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:25.427990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:25.428555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:25.430047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:25.431322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:25.433411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:25.438180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5e3d1-5721-4252-9535-3f3537398873 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:25.462595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:25.463790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:25.463790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:25.464270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:25.465109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:25.468861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:25.482065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 872/1724] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:13:25.483187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:25.521218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa890eb-8936-48fb-b801-7c1267495129 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:25.538843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:25.539679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:25.541239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:25.544122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:25.544122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:25.549323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:25.564214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 853/1725] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:25 2022] POST /identity/v3/projects => generated 367 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:25.569868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:25.640201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:25.653181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 873/1726] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:25 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:25.653181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:25.828038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a641e-4042-4065-bb3b-b0fba6373279 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:25.912012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8a2c9-44b6-4eb9-ae2b-da167820f353 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:25.934860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:25.935737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:25.936138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:25.936802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:25.943623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:25.949120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:26.041544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 874/1727] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:25 2022] POST /identity/v3/users => generated 288 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:26.042831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:26.064646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100d09b-bc6f-4b98-ade3-e88eb292cd28 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:26.067189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:26.068005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.070294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.071237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.073168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:26.075663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:26.092243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f833395-a772-45cd-a740-b700735fe70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:26.094836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f833395-a772-45cd-a740-b700735fe70c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.100213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f833395-a772-45cd-a740-b700735fe70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.100660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f833395-a772-45cd-a740-b700735fe70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.101592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f833395-a772-45cd-a740-b700735fe70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:26.108585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f833395-a772-45cd-a740-b700735fe70c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:26.153483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 875/1728] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:26.168784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c648b1-841d-4a22-a85e-2863d7b0f726 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.169253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 854/1729] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:25 2022] POST /identity/v3/users => generated 290 bytes in 600 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:26.172117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c648b1-841d-4a22-a85e-2863d7b0f726 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.173439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c648b1-841d-4a22-a85e-2863d7b0f726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.173827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:26.227445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dcb0e-0387-401e-9ce6-6374f7b18b4c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:26.255032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:26.255888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.256269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.256715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.257601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:26.259458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:26.260748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f5c648b1-841d-4a22-a85e-2863d7b0f726 None None] MFA Rules not processed for user `754c137a554941d99f7ef9c380a55ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:26.262112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f5c648b1-841d-4a22-a85e-2863d7b0f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:26.308973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 855/1730] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:26.310353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:26.349316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca43073-5a3c-4c0d-9c3d-884389238df4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:26.378414 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:26.379473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.379861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.380253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.384213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:26.386770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:26.444668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:26.472661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 856/1731] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:26.476184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:26.518901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 876/1732] 10.222.0.40 () {58 vars in 1015 bytes} [Tue Jul 12 07:13:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:26.521446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c0dff-1b03-4283-86fe-9c2e9d616114 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:26.532623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c648b1-841d-4a22-a85e-2863d7b0f726 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:26.545273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb5a60-3910-4399-9c1c-09fe1e149b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:26.546701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb5a60-3910-4399-9c1c-09fe1e149b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.548948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb5a60-3910-4399-9c1c-09fe1e149b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.550902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb5a60-3910-4399-9c1c-09fe1e149b66 admin admin] PATH_INFO: `/v3/projects/17fb935be5d14f33b3ca8278b2d923a1/users/a99e047f4af24d5896d5ed40a66f02af/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.563137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c648b1-841d-4a22-a85e-2863d7b0f726 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:26.588008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:26.588827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.589257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.589637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] PATH_INFO: `/v3/projects/24b0555914c74b31a516d2c04dbcb558/users/165bfd90ec01454c9f648e4a9915c116/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.602003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb5a60-3910-4399-9c1c-09fe1e149b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fb935be5d14f33b3ca8278b2d923a1, user_id=a99e047f4af24d5896d5ed40a66f02af, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:26.606681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb5a60-3910-4399-9c1c-09fe1e149b66 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:26.630328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b0555914c74b31a516d2c04dbcb558, user_id=165bfd90ec01454c9f648e4a9915c116, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:26.641114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 857/1733] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:26 2022] PUT /identity/v3/projects/17fb935be5d14f33b3ca8278b2d923a1/users/a99e047f4af24d5896d5ed40a66f02af/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:26.641641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:26.646322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ed43-9183-4fb6-94e4-a3a719115796 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.646999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ed43-9183-4fb6-94e4-a3a719115796 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.647547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ed43-9183-4fb6-94e4-a3a719115796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.677754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 877/1734] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:26 2022] PUT /identity/v3/projects/24b0555914c74b31a516d2c04dbcb558/users/165bfd90ec01454c9f648e4a9915c116/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:26.681965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:26.742653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d025ed43-9183-4fb6-94e4-a3a719115796 None None] MFA Rules not processed for user `a99e047f4af24d5896d5ed40a66f02af`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:26.748382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d025ed43-9183-4fb6-94e4-a3a719115796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:26.932720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dc61e-cc98-4dbd-8a72-db1fc75ff314 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:26.965714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:26.967032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:26.970139 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:26.972803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:26.981730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:26.995736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:27.047964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:27.076540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 878/1735] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:27.077277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 858/1736] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:13:27.078040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:27.079259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ed43-9183-4fb6-94e4-a3a719115796 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:27.085964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:27.119012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ed43-9183-4fb6-94e4-a3a719115796 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:27.138239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:27.139096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:27.139524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:27.139940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:27.140955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:27.145731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:27.219389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 859/1737] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:27 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:27.219389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:27.305835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e929-bc65-4fc4-82e9-d1960cd53df9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:27.480749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2f7c9-ceee-4de5-a8cc-01b9fdd1cc36 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:27.497421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:27.498647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:27.498928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:27.499273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] PATH_INFO: `/v3/projects/24b0555914c74b31a516d2c04dbcb558/users/165bfd90ec01454c9f648e4a9915c116/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:27.501522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b0555914c74b31a516d2c04dbcb558, user_id=165bfd90ec01454c9f648e4a9915c116, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:27.508097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:27.515857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf61ce-3051-4a7d-a919-ca2e79bf7594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:27.516515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf61ce-3051-4a7d-a919-ca2e79bf7594 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:27.516790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf61ce-3051-4a7d-a919-ca2e79bf7594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:27.517091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf61ce-3051-4a7d-a919-ca2e79bf7594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:27.540719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 860/1738] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:27 2022] PUT /identity/v3/projects/24b0555914c74b31a516d2c04dbcb558/users/165bfd90ec01454c9f648e4a9915c116/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:27.541805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:27.547719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf61ce-3051-4a7d-a919-ca2e79bf7594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:27.551692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf61ce-3051-4a7d-a919-ca2e79bf7594 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:27.608586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 879/1739] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:27.610401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:27.610696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:27.610989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:27.713149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] MFA Rules not processed for user `165bfd90ec01454c9f648e4a9915c116`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:27.714405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:27.719332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aece6-c80f-4dc1-ac2e-18a9e95c9f0c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:27.736601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:27.737506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:27.741387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:27.741717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:27.742000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:27.745338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:27.751698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:27.834633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 861/1740] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:27 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:27.837161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:27.876022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d80f35-6df0-44dd-8c25-7be9095b629a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:27.887060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:27.888749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:27.890473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:27.890931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:27.891737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:27.896446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:27.935141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 880/1741] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:27.943441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 862/1742] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:27.944687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:27.945973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:27.947414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:27.983169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bd56a-c7aa-47ca-b866-2ea776f10d45 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:28.012914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:28.016623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:28.018003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:28.019763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] PATH_INFO: `/v3/projects/b83ce3b3b16d47d490a7fe788865fd84/users/a818a43f40e942189d26223cf8f17ff0/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:28.071191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83ce3b3b16d47d490a7fe788865fd84, user_id=a818a43f40e942189d26223cf8f17ff0, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:28.093606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:28.140283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1290f-e0ea-454e-b99c-ee87a7202434 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:28.146587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 863/1743] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:27 2022] PUT /identity/v3/projects/b83ce3b3b16d47d490a7fe788865fd84/users/a818a43f40e942189d26223cf8f17ff0/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:28.154724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:28.201845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9569e8-a2c6-4d51-9e44-4e8bfeba4403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:28.202518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9569e8-a2c6-4d51-9e44-4e8bfeba4403 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:28.202807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9569e8-a2c6-4d51-9e44-4e8bfeba4403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:28.203231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9569e8-a2c6-4d51-9e44-4e8bfeba4403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:28.254636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:28.338487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e13c-5cae-4425-9c5f-f7c0e73b9872 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:28.350517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:28.351215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:28.351516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:28.351773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:28.352515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:28.354149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:28.358135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9569e8-a2c6-4d51-9e44-4e8bfeba4403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:28.366949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9569e8-a2c6-4d51-9e44-4e8bfeba4403 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:28.390019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 864/1744] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:28.391401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 881/1745] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:28.392733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:28.404416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae1687-a5e4-41ac-aef5-89414b0e4e14 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:28.410220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:28.411355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:28.411641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:28.411915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] PATH_INFO: `/v3/projects/b83ce3b3b16d47d490a7fe788865fd84/users/a818a43f40e942189d26223cf8f17ff0/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:28.414379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83ce3b3b16d47d490a7fe788865fd84, user_id=a818a43f40e942189d26223cf8f17ff0, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:28.418871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:28.445953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 865/1746] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:28 2022] PUT /identity/v3/projects/b83ce3b3b16d47d490a7fe788865fd84/users/a818a43f40e942189d26223cf8f17ff0/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:28.458517 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d6781-b6d2-4b72-a225-ff66a6b771d9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:28.458828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d6781-b6d2-4b72-a225-ff66a6b771d9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:28.459099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d6781-b6d2-4b72-a225-ff66a6b771d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:28.535468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:28.551692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3e5d6781-b6d2-4b72-a225-ff66a6b771d9 None None] MFA Rules not processed for user `a818a43f40e942189d26223cf8f17ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:28.555474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3e5d6781-b6d2-4b72-a225-ff66a6b771d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:28.824268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 882/1747] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:28.836845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:29.017931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91041c72-3db0-4635-b64b-c7db95da8d1d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:29.040112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:29.040947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:29.042600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:29.042901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:29.243144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:29.251621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:29.289814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 866/1748] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:32.368835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab0c6-bcfd-47ff-8769-4819e500eb4a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:32.370027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab0c6-bcfd-47ff-8769-4819e500eb4a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:32.370501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab0c6-bcfd-47ff-8769-4819e500eb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:32.426444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-33aab0c6-bcfd-47ff-8769-4819e500eb4a None None] MFA Rules not processed for user `ef6395fa14b54c43931ffeaab023d97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:32.433219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-33aab0c6-bcfd-47ff-8769-4819e500eb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:32.612731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 883/1749] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:32.982139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:33.220711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b23039-8e19-4828-83bf-e2474e5431ff service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:33.239062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:33.239990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:33.243310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:33.244585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] PATH_INFO: `/v3/users/c31f96e10614498d9ad21435767d6cde` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:33.248760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31f96e10614498d9ad21435767d6cde)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:33.251493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eff3e-cc7a-4f0c-a54b-6c9800bde581 None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:33.251975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eff3e-cc7a-4f0c-a54b-6c9800bde581 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:33.252397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eff3e-cc7a-4f0c-a54b-6c9800bde581 None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:33.254234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 884/1750] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 07:13:33.258921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:33.270863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:33.407513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31f96e10614498d9ad21435767d6cde'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:33.412805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31f96e10614498d9ad21435767d6cde'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:33.447379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31f96e10614498d9ad21435767d6cde'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:33.448147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31f96e10614498d9ad21435767d6cde'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:33.465201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8dc201e4-1d59-4dd4-9ac4-de90af75fa41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31f96e10614498d9ad21435767d6cde'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:33.467302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:33.467848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:33.467848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:33.518287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 867/1751] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:32 2022] DELETE /identity/v3/users/c31f96e10614498d9ad21435767d6cde => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:33.531314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:33.531634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:33.531884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:33.536853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:33.538379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:33.650407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:33.656775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:33.826153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 885/1752] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:13:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:13:33.827580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:33.830042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:33.860961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 868/1753] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:34.009087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:34.019836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:34.025174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:34.107364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04dc5-eda1-4eef-ad98-7f7101c3c6f9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:34.124447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:34.128639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:34.130113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:34.130606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] PATH_INFO: `/v3/users/a99e047f4af24d5896d5ed40a66f02af` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:34.137884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99e047f4af24d5896d5ed40a66f02af)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:34.147386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:34.183395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac0be5-ce4a-4879-9c82-f67f0c6f503a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:34.196185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee4a96-15c0-4198-b7fd-8d0d96b24b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:34.200211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee4a96-15c0-4198-b7fd-8d0d96b24b05 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:34.200658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee4a96-15c0-4198-b7fd-8d0d96b24b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:34.203370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee4a96-15c0-4198-b7fd-8d0d96b24b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:34.232979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee4a96-15c0-4198-b7fd-8d0d96b24b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:34.237424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee4a96-15c0-4198-b7fd-8d0d96b24b05 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:34.282893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 869/1754] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:13:34.293766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99e047f4af24d5896d5ed40a66f02af'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:34.296433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99e047f4af24d5896d5ed40a66f02af'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:34.300457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99e047f4af24d5896d5ed40a66f02af'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:34.341274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99e047f4af24d5896d5ed40a66f02af'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:34.366146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99e047f4af24d5896d5ed40a66f02af'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:34.407670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 886/1755] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:33 2022] DELETE /identity/v3/users/a99e047f4af24d5896d5ed40a66f02af => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:34.413461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:34.413764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:34.414033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:34.441421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:34.442863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:34.469866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:34.642190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 870/1756] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:34.843984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:34.847403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:35.036632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af413-cf65-4553-a86e-72044215a96a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:35.041115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:35.041864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:35.042209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:35.042504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:35.068347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:35.072560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:35.100084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 887/1757] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:36.183048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:36.184500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:36.288505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482c873-5690-42f5-8833-c815fb7c1592 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:36.301540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1677f4-cd6b-4c74-b333-35be351ffa59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:36.302252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1677f4-cd6b-4c74-b333-35be351ffa59 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:36.302533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1677f4-cd6b-4c74-b333-35be351ffa59 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:36.302815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1677f4-cd6b-4c74-b333-35be351ffa59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:36.305030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2a1677f4-cd6b-4c74-b333-35be351ffa59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:36.358074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1677f4-cd6b-4c74-b333-35be351ffa59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:36.364241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1677f4-cd6b-4c74-b333-35be351ffa59 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:36.386413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 871/1758] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:36.769654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:36.771729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:36.804638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4afb7c-d996-4b29-a5ef-0c4078540655 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:36.841558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:36.842501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:36.842781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:36.843055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] PATH_INFO: `/v3/users/8b28e89a0173461aa5faefc14948c616` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:36.847928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b28e89a0173461aa5faefc14948c616)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:36.850150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:36.957944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b28e89a0173461aa5faefc14948c616'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:36.958921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b28e89a0173461aa5faefc14948c616'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:36.960342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b28e89a0173461aa5faefc14948c616'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:36.988813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:37.017067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b28e89a0173461aa5faefc14948c616'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.034990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b28e89a0173461aa5faefc14948c616'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.064786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 888/1759] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:36 2022] DELETE /identity/v3/users/8b28e89a0173461aa5faefc14948c616 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:13:37.071079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:37.072446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:37.076610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:37.124968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:37.131833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:37.300993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:37.310136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:37.320692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 872/1760] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:37.327896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda4b8a-5eb7-4a64-b1cd-ef5ec4a7aab4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:37.334123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:37.338180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:37.338673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:37.339088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] PATH_INFO: `/v3/projects/17fb935be5d14f33b3ca8278b2d923a1` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:37.347234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fb935be5d14f33b3ca8278b2d923a1)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:37.349433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:37.445402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:37.446831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:37.495746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fb935be5d14f33b3ca8278b2d923a1', 'request_id': 'req-748afe3b-44fd-4a28-9959-dc616195dc06'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.517847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fb935be5d14f33b3ca8278b2d923a1', 'request_id': 'req-748afe3b-44fd-4a28-9959-dc616195dc06'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.537571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fb935be5d14f33b3ca8278b2d923a1', 'request_id': 'req-748afe3b-44fd-4a28-9959-dc616195dc06'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.566467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] The token cache is being invalidate because project 17fb935be5d14f33b3ca8278b2d923a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:37.568070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.571660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.577972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 889/1761] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:37 2022] DELETE /identity/v3/projects/17fb935be5d14f33b3ca8278b2d923a1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:37.579191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:37.580669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:37.621813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b8483-d577-4322-bfd6-5b3f38b0595e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:37.639287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:37.794948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748afe3b-44fd-4a28-9959-dc616195dc06 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:37.814733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91271da9-f932-46d0-b935-2847c69211a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:37.815567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271da9-f932-46d0-b935-2847c69211a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:37.816772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271da9-f932-46d0-b935-2847c69211a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:37.816966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271da9-f932-46d0-b935-2847c69211a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:37.817505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:37.818991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:37.819275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:37.820856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] PATH_INFO: `/v3/users/a818a43f40e942189d26223cf8f17ff0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:37.822651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a818a43f40e942189d26223cf8f17ff0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:37.828406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:37.830382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-91271da9-f932-46d0-b935-2847c69211a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:37.862966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91271da9-f932-46d0-b935-2847c69211a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:37.864862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91271da9-f932-46d0-b935-2847c69211a8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:37.901859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 873/1762] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:37.959925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a818a43f40e942189d26223cf8f17ff0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.964452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a818a43f40e942189d26223cf8f17ff0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.965580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a818a43f40e942189d26223cf8f17ff0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:37.982821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:38.005399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a818a43f40e942189d26223cf8f17ff0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:38.026494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-16fdb182-e8d9-4412-b41c-0856d4eaef0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a818a43f40e942189d26223cf8f17ff0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:38.062268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 890/1763] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:37 2022] DELETE /identity/v3/users/a818a43f40e942189d26223cf8f17ff0 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:38.183656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d7930-e80d-4847-bb2b-ad920a0e2daa None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:38.184022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d7930-e80d-4847-bb2b-ad920a0e2daa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:38.184310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d7930-e80d-4847-bb2b-ad920a0e2daa None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:38.186174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 874/1764] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:13:38.197547 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f729e-6b07-4efd-9993-d5b129eff006 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:38.198019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f729e-6b07-4efd-9993-d5b129eff006 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:38.198410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f729e-6b07-4efd-9993-d5b129eff006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:38.268946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-316f729e-6b07-4efd-9993-d5b129eff006 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:38.274852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-316f729e-6b07-4efd-9993-d5b129eff006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:38.338498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d7930-e80d-4847-bb2b-ad920a0e2daa None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:38.341721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-594d7930-e80d-4847-bb2b-ad920a0e2daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:38.538228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d7930-e80d-4847-bb2b-ad920a0e2daa None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:38.558151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:38.559149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:38.560932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:38.561337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] PATH_INFO: `/v3/projects/353c98633dc14ea49f5134d3477223e8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:38.562710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=353c98633dc14ea49f5134d3477223e8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:38.572097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:38.599285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 891/1765] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:13:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:38.697864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353c98633dc14ea49f5134d3477223e8', 'request_id': 'req-e8513692-0fdb-4105-9f4e-22e530981c6a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:38.730416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353c98633dc14ea49f5134d3477223e8', 'request_id': 'req-e8513692-0fdb-4105-9f4e-22e530981c6a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:38.742447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353c98633dc14ea49f5134d3477223e8', 'request_id': 'req-e8513692-0fdb-4105-9f4e-22e530981c6a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:38.761211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] The token cache is being invalidate because project 353c98633dc14ea49f5134d3477223e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:38.761608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:38.762354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:38.765540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 875/1766] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:38 2022] DELETE /identity/v3/projects/353c98633dc14ea49f5134d3477223e8 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:13:38.771206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f729e-6b07-4efd-9993-d5b129eff006 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:38.784896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f729e-6b07-4efd-9993-d5b129eff006 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:38.974329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:38.976111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:38.976659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:38.977094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] PATH_INFO: `/v3/users/165bfd90ec01454c9f648e4a9915c116` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:38.981929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=165bfd90ec01454c9f648e4a9915c116)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:38.984311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:39.123203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '165bfd90ec01454c9f648e4a9915c116'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.124425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '165bfd90ec01454c9f648e4a9915c116'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.125280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '165bfd90ec01454c9f648e4a9915c116'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.155653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '165bfd90ec01454c9f648e4a9915c116'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.157436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:39.170040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '165bfd90ec01454c9f648e4a9915c116'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.215147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 892/1767] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:38 2022] DELETE /identity/v3/users/165bfd90ec01454c9f648e4a9915c116 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:39.470235 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:39.473026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:39.553238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:39.593068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8513692-0fdb-4105-9f4e-22e530981c6a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:39.683856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3ad7c-3553-4da8-8376-dc010a79f15b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:39.684580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3ad7c-3553-4da8-8376-dc010a79f15b service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:39.684861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3ad7c-3553-4da8-8376-dc010a79f15b service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:39.685757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3ad7c-3553-4da8-8376-dc010a79f15b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:39.689186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3fb3ad7c-3553-4da8-8376-dc010a79f15b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:39.703604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3ad7c-3553-4da8-8376-dc010a79f15b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:39.705242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3ad7c-3553-4da8-8376-dc010a79f15b service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:39.717101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 876/1768] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:39.731200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:39.732648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:39.803740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e53ddf-db19-47ef-ad2a-1e9041b95d08 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:39.809326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:39.810484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:39.810950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:39.811491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] PATH_INFO: `/v3/projects/b83ce3b3b16d47d490a7fe788865fd84` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:39.812757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83ce3b3b16d47d490a7fe788865fd84)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:39.814715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:39.871221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83ce3b3b16d47d490a7fe788865fd84', 'request_id': 'req-83015637-3baf-41a3-b2b4-d274b14f4502'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.882446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83ce3b3b16d47d490a7fe788865fd84', 'request_id': 'req-83015637-3baf-41a3-b2b4-d274b14f4502'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.889567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83ce3b3b16d47d490a7fe788865fd84', 'request_id': 'req-83015637-3baf-41a3-b2b4-d274b14f4502'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.912312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] The token cache is being invalidate because project b83ce3b3b16d47d490a7fe788865fd84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:39.912691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.913605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:39.915267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 893/1769] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:39 2022] DELETE /identity/v3/projects/b83ce3b3b16d47d490a7fe788865fd84 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:39.921606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268a8aa-ab50-469a-8ebf-61b89eac578c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:39.922473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268a8aa-ab50-469a-8ebf-61b89eac578c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:39.922473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268a8aa-ab50-469a-8ebf-61b89eac578c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:39.929387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:39.932808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:39.971770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8268a8aa-ab50-469a-8ebf-61b89eac578c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:39.973079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8268a8aa-ab50-469a-8ebf-61b89eac578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.114027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 877/1770] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:40.121695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b4f0-cf01-47a2-991c-177cddc28ab6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.122109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b4f0-cf01-47a2-991c-177cddc28ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.122613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b4f0-cf01-47a2-991c-177cddc28ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.133943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83015637-3baf-41a3-b2b4-d274b14f4502 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.143605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.144505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.144955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.145266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] PATH_INFO: `/v3/projects/466c236ee03c43b995d9f8c2849f74c9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.146751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466c236ee03c43b995d9f8c2849f74c9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.148686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.155977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-52e0b4f0-cf01-47a2-991c-177cddc28ab6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:40.157306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-52e0b4f0-cf01-47a2-991c-177cddc28ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.165283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:40.166229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 878/1771] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:13:40.169991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0b4f0-cf01-47a2-991c-177cddc28ab6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.180343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0b4f0-cf01-47a2-991c-177cddc28ab6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.185877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.186594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.187396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.187396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.188110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:40.188418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.190630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.203637 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 879/1772] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:13:40.207616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.213592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466c236ee03c43b995d9f8c2849f74c9', 'request_id': 'req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.220985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca4a0e-6aad-4e0c-b433-4ca104aacfd4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.225893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466c236ee03c43b995d9f8c2849f74c9', 'request_id': 'req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.232117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466c236ee03c43b995d9f8c2849f74c9', 'request_id': 'req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.254524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] The token cache is being invalidate because project 466c236ee03c43b995d9f8c2849f74c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:40.254788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.255586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.257543 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 894/1773] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:39 2022] DELETE /identity/v3/projects/466c236ee03c43b995d9f8c2849f74c9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:13:40.258108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.259647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.282053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:40.304535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fa9cc-70c6-4076-8a0a-d448f63d922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.305746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fa9cc-70c6-4076-8a0a-d448f63d922e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.306333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fa9cc-70c6-4076-8a0a-d448f63d922e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.306794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fa9cc-70c6-4076-8a0a-d448f63d922e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.307700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0fa9cc-70c6-4076-8a0a-d448f63d922e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.309747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0fa9cc-70c6-4076-8a0a-d448f63d922e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.338606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb712b4-0f55-4b1d-84be-5a31cc7e31a9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.344976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 880/1774] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:40.344976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.344976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.344976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.347141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.348079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.348364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.348640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] PATH_INFO: `/v3/projects/24b0555914c74b31a516d2c04dbcb558` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.349914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b0555914c74b31a516d2c04dbcb558)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.351838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.365781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:40.366913 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.411417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b0555914c74b31a516d2c04dbcb558', 'request_id': 'req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.418528 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b0555914c74b31a516d2c04dbcb558', 'request_id': 'req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.425695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b0555914c74b31a516d2c04dbcb558', 'request_id': 'req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.440030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 881/1775] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:40.440718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.446403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] The token cache is being invalidate because project 24b0555914c74b31a516d2c04dbcb558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:40.446744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.447889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-eb0a8ecd-9df9-4718-931b-9e76067aa6a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:40.452809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 895/1776] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:40 2022] DELETE /identity/v3/projects/24b0555914c74b31a516d2c04dbcb558 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:40.453890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.454155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cbb6c-bb6c-49ee-be93-a717574dc7ad None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.454446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cbb6c-bb6c-49ee-be93-a717574dc7ad None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.454724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cbb6c-bb6c-49ee-be93-a717574dc7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.473540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92c519-a192-4f96-a331-63f3baf5addc None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.475930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-680cbb6c-bb6c-49ee-be93-a717574dc7ad None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:40.477243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-680cbb6c-bb6c-49ee-be93-a717574dc7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.511969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:40.548811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.550399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.550679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.551010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.551713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.553878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.565074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 896/1777] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:40.566585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.566891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.567863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.584399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 882/1778] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:40.584399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.593633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5227f05-beb9-4ff3-9e34-c89bb4cbc85d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.594486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:40.596489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.599245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeeadb-24d6-4c4d-8687-9e59bbd67b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.600041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeeadb-24d6-4c4d-8687-9e59bbd67b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.600387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeeadb-24d6-4c4d-8687-9e59bbd67b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.600758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeeadb-24d6-4c4d-8687-9e59bbd67b49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.601656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeeadb-24d6-4c4d-8687-9e59bbd67b49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:40.602027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeeadb-24d6-4c4d-8687-9e59bbd67b49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.603703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeeadb-24d6-4c4d-8687-9e59bbd67b49 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.606476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 897/1779] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:40.607437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.609326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.623473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aea80d-9cb5-4c0c-a0dc-c86440c96712 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.626017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 883/1780] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:13:40.627576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9a7b3-2521-43d9-a2e0-1551e8f02906 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.627933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9a7b3-2521-43d9-a2e0-1551e8f02906 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.628302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9a7b3-2521-43d9-a2e0-1551e8f02906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.632891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.633961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.634266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.634572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.635281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.636902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.648363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-86c9a7b3-2521-43d9-a2e0-1551e8f02906 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:40.652007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 898/1781] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:40.652007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.652242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-86c9a7b3-2521-43d9-a2e0-1551e8f02906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.660577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 884/1782] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:40.660970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0cc9a-cb44-4a2b-addc-9b59fe038ba9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.661442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9a7b3-2521-43d9-a2e0-1551e8f02906 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.668403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.669251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.669537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.669807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.670859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.672784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.673962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9a7b3-2521-43d9-a2e0-1551e8f02906 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.678712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.679381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.679660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.679932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.680700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.682319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.692892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 885/1783] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:40.697101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.701126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 899/1784] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:40.701267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.707380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9dac2-1a97-4cea-9da2-5eb4abea3d4e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.732110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:40.766933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.767673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.768041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.768321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.769095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:40.769406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.771014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.775161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57528e9-3267-46a3-a2a2-7cfa77afd0cc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.780118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.781096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.781503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.781788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] PATH_INFO: `/v3/projects/0948031a7ee942369973933ff2790e4a/users/0c1ee9a47f72405b90a35b8090a20a0d/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.785342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 886/1785] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:13:40.786249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.798161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed769ea-a066-4c8f-add8-317f3d7f79d6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.800293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0948031a7ee942369973933ff2790e4a, user_id=0c1ee9a47f72405b90a35b8090a20a0d, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.803431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd964b14-c6d7-412b-8b48-9159f1214bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.803586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.804062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd964b14-c6d7-412b-8b48-9159f1214bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.804375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd964b14-c6d7-412b-8b48-9159f1214bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.804734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd964b14-c6d7-412b-8b48-9159f1214bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.805387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd964b14-c6d7-412b-8b48-9159f1214bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.807270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd964b14-c6d7-412b-8b48-9159f1214bd9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.818309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 900/1786] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:40 2022] PUT /identity/v3/projects/0948031a7ee942369973933ff2790e4a/users/0c1ee9a47f72405b90a35b8090a20a0d/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:40.819337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.829916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 887/1787] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:40.831372 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80b1ea-d0d8-481e-9217-b23e05a8125b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.831569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80b1ea-d0d8-481e-9217-b23e05a8125b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.831848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80b1ea-d0d8-481e-9217-b23e05a8125b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.864933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0d80b1ea-d0d8-481e-9217-b23e05a8125b None None] MFA Rules not processed for user `0c1ee9a47f72405b90a35b8090a20a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:40.866228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0d80b1ea-d0d8-481e-9217-b23e05a8125b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.902121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a000a67-f031-40f7-88b7-3c6ef537fd3a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.907834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:40.908620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:40.909021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:40.909425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:40.910319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:40.912835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:40.945326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 888/1788] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:40.945326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80b1ea-d0d8-481e-9217-b23e05a8125b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.950969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 901/1789] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:40 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:40.953978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:40.957885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80b1ea-d0d8-481e-9217-b23e05a8125b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:40.962177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:40.969614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:41.036837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.038053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.038483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.039531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.040422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.042742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.050576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9ee32-5a42-43d5-a1b9-d0330f9a1428 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.055491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.056375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.056865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.057282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 889/1790] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:41.057475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.058185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.070823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b9df-a96c-4090-90bc-dccf57968e9f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.078070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41add506-9c58-4644-9391-620121d83590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.078758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41add506-9c58-4644-9391-620121d83590 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.079039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41add506-9c58-4644-9391-620121d83590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.079358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41add506-9c58-4644-9391-620121d83590 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.080059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41add506-9c58-4644-9391-620121d83590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.082473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41add506-9c58-4644-9391-620121d83590 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.119954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 890/1791] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:41 2022] POST /identity/v3/users => generated 294 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:41.120902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41add506-9c58-4644-9391-620121d83590 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.134477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41add506-9c58-4644-9391-620121d83590 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.140029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.140647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.140932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.141426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.143111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.144949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.146305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.148853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.173457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 902/1792] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:41.174601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.181072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 891/1793] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:41.182673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.193367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:41.204871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514152d-62bb-4db6-ad6e-ee086f584a79 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.207650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfe61a-2972-42fd-936e-e1e4fb651f89 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.217700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.218376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.218658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.218933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.220037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.221828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.222212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36b649-7321-4f81-9548-427f78d9ec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.226228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36b649-7321-4f81-9548-427f78d9ec0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.226545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36b649-7321-4f81-9548-427f78d9ec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.226906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36b649-7321-4f81-9548-427f78d9ec0f admin admin] PATH_INFO: `/v3/projects/1bffe8e53f534c1ba596feccd9de7acb/users/be595828d401488d9390f584be914376/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.233983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 903/1794] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:41.238147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.256411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36b649-7321-4f81-9548-427f78d9ec0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bffe8e53f534c1ba596feccd9de7acb, user_id=be595828d401488d9390f584be914376, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.259624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cc079-218c-4dd4-84ce-e2d91e1e250a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.262020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36b649-7321-4f81-9548-427f78d9ec0f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.267276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.268296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.268745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.269818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.270096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.274077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.292717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 904/1795] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:41.292930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 892/1796] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:41 2022] PUT /identity/v3/projects/1bffe8e53f534c1ba596feccd9de7acb/users/be595828d401488d9390f584be914376/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:41.293852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.302581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a344a91-37e4-475d-b3d3-eaf9e588014b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.305123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a344a91-37e4-475d-b3d3-eaf9e588014b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.305425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a344a91-37e4-475d-b3d3-eaf9e588014b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.319914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cd0b7-aefa-40e5-90c0-662f69910bb1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.358653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6a344a91-37e4-475d-b3d3-eaf9e588014b None None] MFA Rules not processed for user `be595828d401488d9390f584be914376`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:41.360165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6a344a91-37e4-475d-b3d3-eaf9e588014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:41.423496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba62099-d691-4d1a-b303-7657b324474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.424950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba62099-d691-4d1a-b303-7657b324474b admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.425369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba62099-d691-4d1a-b303-7657b324474b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.426741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba62099-d691-4d1a-b303-7657b324474b admin admin] PATH_INFO: `/v3/projects/62028edb09ac4a50987b17d79b728af9/users/3a4bd68aa038431db58561da144e5072/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.454095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba62099-d691-4d1a-b303-7657b324474b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62028edb09ac4a50987b17d79b728af9, user_id=3a4bd68aa038431db58561da144e5072, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.458401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba62099-d691-4d1a-b303-7657b324474b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.485101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 905/1797] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:41 2022] PUT /identity/v3/projects/62028edb09ac4a50987b17d79b728af9/users/3a4bd68aa038431db58561da144e5072/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:41.492186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195633e8-3322-4add-b1b4-b01715339087 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.492608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195633e8-3322-4add-b1b4-b01715339087 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.493262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195633e8-3322-4add-b1b4-b01715339087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.497626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 893/1798] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:41.502011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a344a91-37e4-475d-b3d3-eaf9e588014b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.513396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a344a91-37e4-475d-b3d3-eaf9e588014b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.537010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-195633e8-3322-4add-b1b4-b01715339087 None None] MFA Rules not processed for user `3a4bd68aa038431db58561da144e5072`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:41.538559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-195633e8-3322-4add-b1b4-b01715339087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:41.547289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:41.579465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.580163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.580756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.580921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.581705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.583861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.618514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 894/1799] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:41 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:13:41.620039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.684740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 906/1800] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:41.685905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195633e8-3322-4add-b1b4-b01715339087 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.703492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:41.715062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195633e8-3322-4add-b1b4-b01715339087 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.720101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223bf4b-1a5e-49c9-a3ce-efa46478b5dc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.725753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.726430 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.726692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.726973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.727731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.728899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.729744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.729859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.730196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.731342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.733499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.736469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.772019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 907/1801] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:41 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:41.773998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.779973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 895/1802] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:41 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:41.784059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.803829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02940d-30e1-4f3a-a24e-0a075e04417d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.880560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde6d1-1b1e-4eb9-ad93-ce505c7429de admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.884075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.884995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.885477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.885896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.888152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.888472 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.889291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.890958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.890958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.890958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.892227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.894643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.906862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 896/1803] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:41.912148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.925888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1624e7f-204d-4956-a9ef-bf2672eba55c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.931290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.934566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 908/1804] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:41 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:41.934713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:41.934713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:41.934713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] PATH_INFO: `/v3/projects/a73153785f1f4d8ea88a9e18a86a77b2/users/5bbac03acca549158f3605f545659ffa/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:41.937477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:41.959874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73153785f1f4d8ea88a9e18a86a77b2, user_id=5bbac03acca549158f3605f545659ffa, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:41.964074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:41.967588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da334a4-4a59-47f0-8a2a-ecff778542c1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:41.986117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:41.999268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 897/1805] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:41 2022] PUT /identity/v3/projects/a73153785f1f4d8ea88a9e18a86a77b2/users/5bbac03acca549158f3605f545659ffa/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:13:42.001195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.001513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.001801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.003830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.005214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.009612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.034860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 909/1806] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:42.041462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.063965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc120e-0cf7-486e-ac22-5664e3cd14e8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.090804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc75bd9-a87f-4fb2-b3b4-652fb249b595 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.099032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.099823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.100168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.100703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.101610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.104066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.121239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 898/1807] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:42.122261 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.141500 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f20e31-4ada-4382-a8a6-4d52f64f3fbb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.150041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb382f1-627a-419c-8c75-78e8c9bf5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.151295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb382f1-627a-419c-8c75-78e8c9bf5fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.151773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb382f1-627a-419c-8c75-78e8c9bf5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.152192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb382f1-627a-419c-8c75-78e8c9bf5fc1 admin admin] PATH_INFO: `/v3/projects/a73153785f1f4d8ea88a9e18a86a77b2/users/5bbac03acca549158f3605f545659ffa/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.153798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.154491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.154827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.155102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.155995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.156674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb382f1-627a-419c-8c75-78e8c9bf5fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73153785f1f4d8ea88a9e18a86a77b2, user_id=5bbac03acca549158f3605f545659ffa, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.157727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.161082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb382f1-627a-419c-8c75-78e8c9bf5fc1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.163654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:42.177494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 910/1808] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:42.178497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.191174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291e79d-2fd5-4a87-8b99-f2277d35bf4c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.192591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 899/1809] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:42 2022] PUT /identity/v3/projects/a73153785f1f4d8ea88a9e18a86a77b2/users/5bbac03acca549158f3605f545659ffa/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:42.194496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502bbb-8865-4024-831b-eb7bea3616f0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.195008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502bbb-8865-4024-831b-eb7bea3616f0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.195427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502bbb-8865-4024-831b-eb7bea3616f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.236373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f2502bbb-8865-4024-831b-eb7bea3616f0 None None] MFA Rules not processed for user `5bbac03acca549158f3605f545659ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:42.237788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f2502bbb-8865-4024-831b-eb7bea3616f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:42.271304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:42.305345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f884c-b7fa-49ca-af20-deb476519532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.307224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f884c-b7fa-49ca-af20-deb476519532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.307519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f884c-b7fa-49ca-af20-deb476519532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.307798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f884c-b7fa-49ca-af20-deb476519532 admin admin] PATH_INFO: `/v3/projects/f5a54dd42c53452db63a8f8ac5dd45a3/users/f96cb5a040d3486e948a8f2a6ff85a25/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.339083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f884c-b7fa-49ca-af20-deb476519532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a54dd42c53452db63a8f8ac5dd45a3, user_id=f96cb5a040d3486e948a8f2a6ff85a25, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.342750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f884c-b7fa-49ca-af20-deb476519532 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.373340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 911/1810] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:42 2022] PUT /identity/v3/projects/f5a54dd42c53452db63a8f8ac5dd45a3/users/f96cb5a040d3486e948a8f2a6ff85a25/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:42.386397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c81a6-8a48-40cb-ae3a-78e88f7438fe None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.386715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c81a6-8a48-40cb-ae3a-78e88f7438fe None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.388199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c81a6-8a48-40cb-ae3a-78e88f7438fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.389895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 900/1811] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:42.402189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502bbb-8865-4024-831b-eb7bea3616f0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.440835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ee4c81a6-8a48-40cb-ae3a-78e88f7438fe None None] MFA Rules not processed for user `f96cb5a040d3486e948a8f2a6ff85a25`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:42.443168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ee4c81a6-8a48-40cb-ae3a-78e88f7438fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:42.491270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502bbb-8865-4024-831b-eb7bea3616f0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.561040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 912/1812] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:13:42.563136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.563850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.564130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.564400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.564510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c81a6-8a48-40cb-ae3a-78e88f7438fe None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.625598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.627760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.632523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c81a6-8a48-40cb-ae3a-78e88f7438fe None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.637423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.638101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.638386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.638661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.639362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.641303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.645633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 901/1813] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:13:42.664513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:42.678508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 913/1814] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:42 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:42.679727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.694054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87dab9-3c09-4921-91ed-7e037655342d service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.747435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:42.793441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.794392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.796011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.796716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.797914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.801860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.839149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 902/1815] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:42 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:42.843424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.858656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4562358-7fe7-4e6b-8a05-38227c7fc4d3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.864810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.865666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.866380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.866669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.867892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.869893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.897565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 914/1816] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:42.901599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.915058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd7f65-858d-49cf-95aa-3a35d72b477c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:42.921795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:42.922972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:42.923494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:42.924051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] PATH_INFO: `/v3/projects/9bd1eafb9b5c470aa77945302ae459de/users/a353a9ecf9ea4ebb97817a2269fd1f82/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:42.946520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1eafb9b5c470aa77945302ae459de, user_id=a353a9ecf9ea4ebb97817a2269fd1f82, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:42.949833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:42.975414 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 903/1817] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:42 2022] PUT /identity/v3/projects/9bd1eafb9b5c470aa77945302ae459de/users/a353a9ecf9ea4ebb97817a2269fd1f82/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:42.981030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:42.996068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf631aa-c5bc-486f-a3c0-e5464c37d181 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:43.103312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93ebb5-da07-4591-9dcb-d1816e565440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:43.104370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93ebb5-da07-4591-9dcb-d1816e565440 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:43.104792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93ebb5-da07-4591-9dcb-d1816e565440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:43.105628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93ebb5-da07-4591-9dcb-d1816e565440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:43.106925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93ebb5-da07-4591-9dcb-d1816e565440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:43.111751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93ebb5-da07-4591-9dcb-d1816e565440 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:43.131754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 915/1818] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:43.133035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:43.151893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4147f-0d5f-4e10-a98c-24416dc3c053 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:43.157680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:43.158986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:43.159514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:43.159897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] PATH_INFO: `/v3/projects/9bd1eafb9b5c470aa77945302ae459de/users/a353a9ecf9ea4ebb97817a2269fd1f82/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:43.163453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1eafb9b5c470aa77945302ae459de, user_id=a353a9ecf9ea4ebb97817a2269fd1f82, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:43.169467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:43.185922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 904/1819] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:43 2022] PUT /identity/v3/projects/9bd1eafb9b5c470aa77945302ae459de/users/a353a9ecf9ea4ebb97817a2269fd1f82/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:43.196631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:43.197167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:43.197167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:43.227530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] MFA Rules not processed for user `a353a9ecf9ea4ebb97817a2269fd1f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:43.228810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:43.330550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 916/1820] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:43.342149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:43.421374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61117e71-3f40-446f-8407-9faed3213be0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:43.425802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:43.426576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:43.426924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:43.427281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:43.440083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:43.441851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:43.455491 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 905/1821] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:44.047579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:44.049098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:44.063695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750acb14-9ba3-4368-8252-9c1697bc4587 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:44.070026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151392d-953e-4bc0-b1f8-15f8dffc4c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:44.070829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151392d-953e-4bc0-b1f8-15f8dffc4c71 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:44.071116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151392d-953e-4bc0-b1f8-15f8dffc4c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:44.071398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151392d-953e-4bc0-b1f8-15f8dffc4c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:44.167108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:44.180365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151392d-953e-4bc0-b1f8-15f8dffc4c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:44.182059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151392d-953e-4bc0-b1f8-15f8dffc4c71 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:44.206627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 917/1822] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:44.386177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8517bb-43cc-45db-9de5-27e706bbc535 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:44.391655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:44.397498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:44.397792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:44.398074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] PATH_INFO: `/v3/users/0c1ee9a47f72405b90a35b8090a20a0d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:44.399433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1ee9a47f72405b90a35b8090a20a0d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:44.401892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:44.488807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:44.527016 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1ee9a47f72405b90a35b8090a20a0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:44.528045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1ee9a47f72405b90a35b8090a20a0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:44.566709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1ee9a47f72405b90a35b8090a20a0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:44.567612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1ee9a47f72405b90a35b8090a20a0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:44.590313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1ee9a47f72405b90a35b8090a20a0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:44.630011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:44.630440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:44.630739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:44.633035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 906/1823] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:44 2022] DELETE /identity/v3/users/0c1ee9a47f72405b90a35b8090a20a0d => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:44.655045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:44.671359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:44.674696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:44.808805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 918/1824] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:44.821392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:44.822902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:44.903536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977a09-46df-43d5-a3d4-7ccab6fbba36 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:44.907563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc305add-5575-48a8-84fc-6a456174f85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:44.908650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305add-5575-48a8-84fc-6a456174f85b service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:44.908996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305add-5575-48a8-84fc-6a456174f85b service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:44.909391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305add-5575-48a8-84fc-6a456174f85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:44.919488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc305add-5575-48a8-84fc-6a456174f85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:44.921380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc305add-5575-48a8-84fc-6a456174f85b service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:44.932177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 907/1825] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:45.368096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:45.369583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:45.382854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdd27d-c0c7-455a-b014-33b582196c68 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:45.388066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:45.388952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.389259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.389531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] PATH_INFO: `/v3/projects/0948031a7ee942369973933ff2790e4a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.390741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0948031a7ee942369973933ff2790e4a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:45.392679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:45.444763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0948031a7ee942369973933ff2790e4a', 'request_id': 'req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:45.450878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0948031a7ee942369973933ff2790e4a', 'request_id': 'req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:45.456692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0948031a7ee942369973933ff2790e4a', 'request_id': 'req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:45.470221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] The token cache is being invalidate because project 0948031a7ee942369973933ff2790e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:45.470503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:45.471228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8d04ca09-cf4a-4222-b6a6-8a7643be07b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:45.472869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 919/1826] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:45 2022] DELETE /identity/v3/projects/0948031a7ee942369973933ff2790e4a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:45.477920 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50f476-6b4f-42bd-872b-0effd47fb0ef None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.478426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50f476-6b4f-42bd-872b-0effd47fb0ef None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.478426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50f476-6b4f-42bd-872b-0effd47fb0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.497198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9d50f476-6b4f-42bd-872b-0effd47fb0ef None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:45.498341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9d50f476-6b4f-42bd-872b-0effd47fb0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:45.558955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 908/1827] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:45.566864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f906231-fe1a-4217-8036-2465ca96b1c2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.567149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f906231-fe1a-4217-8036-2465ca96b1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.567420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f906231-fe1a-4217-8036-2465ca96b1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.585695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9f906231-fe1a-4217-8036-2465ca96b1c2 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:45.586876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9f906231-fe1a-4217-8036-2465ca96b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:45.595196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 920/1828] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:45.599370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50f476-6b4f-42bd-872b-0effd47fb0ef None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:45.609145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50f476-6b4f-42bd-872b-0effd47fb0ef None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:45.614636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:45.615152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.615492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.615710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.616432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:45.616730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:45.618339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:45.630454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 909/1829] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:13:45.635012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f906231-fe1a-4217-8036-2465ca96b1c2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:45.646503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f906231-fe1a-4217-8036-2465ca96b1c2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:45.652976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:45.653699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.653979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.654255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.654958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:45.656882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:45.685137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 921/1830] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:45 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:45.689330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:45.699870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21501ce8-b66d-4852-97c0-b636b8819ba8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:45.735216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:45.765344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:45.766091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.766397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.766672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.767385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:45.769620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:45.795748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 910/1831] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:45 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:13:45.799651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:45.813353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b95dae-828d-498b-84fb-4a4208040dda admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:45.820319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:45.821203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.821723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.822138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.823781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:45.826323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:45.839322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 922/1832] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:45.843659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:45.853916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e62a08-e315-4fdd-9bce-62af4f5d0415 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:45.858419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8017c-cfe2-476f-bb38-c3538213a521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:45.859083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8017c-cfe2-476f-bb38-c3538213a521 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.859361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8017c-cfe2-476f-bb38-c3538213a521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.859637 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8017c-cfe2-476f-bb38-c3538213a521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.860358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8017c-cfe2-476f-bb38-c3538213a521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:45.862057 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8017c-cfe2-476f-bb38-c3538213a521 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:45.872573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 911/1833] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:13:45.876651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:45.887627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c53f3e-d9d0-4639-9eb2-cb15a3905df8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:45.892372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:45.893372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.893799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.894076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] PATH_INFO: `/v3/projects/7c646f4279764cc58afaf18685851707/users/95cd3336f6354d3bbe7aaf087baa3e3b/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.911029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c646f4279764cc58afaf18685851707, user_id=95cd3336f6354d3bbe7aaf087baa3e3b, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:45.913815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:45.932609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 923/1834] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:45 2022] PUT /identity/v3/projects/7c646f4279764cc58afaf18685851707/users/95cd3336f6354d3bbe7aaf087baa3e3b/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:45.942779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418b823-5d5b-4183-ad00-c9361848c08d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:45.943100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418b823-5d5b-4183-ad00-c9361848c08d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:45.943371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418b823-5d5b-4183-ad00-c9361848c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:45.969282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b418b823-5d5b-4183-ad00-c9361848c08d None None] MFA Rules not processed for user `95cd3336f6354d3bbe7aaf087baa3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:45.970515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b418b823-5d5b-4183-ad00-c9361848c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:46.041330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 912/1835] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:46.053190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:46.055076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:46.090314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:46.122243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18382b73-7161-44a3-b200-3fa831987078 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:46.125905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8af129-42f7-40f9-85bd-127b812f904a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:46.126558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8af129-42f7-40f9-85bd-127b812f904a service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:46.126838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8af129-42f7-40f9-85bd-127b812f904a service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:46.127137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8af129-42f7-40f9-85bd-127b812f904a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:46.135708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8af129-42f7-40f9-85bd-127b812f904a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:46.137165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8af129-42f7-40f9-85bd-127b812f904a service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:46.146491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 924/1836] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:46.171148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418b823-5d5b-4183-ad00-c9361848c08d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:46.237811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418b823-5d5b-4183-ad00-c9361848c08d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:46.242573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:46.243463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:46.243740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:46.244009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] PATH_INFO: `/v3/users/95cd3336f6354d3bbe7aaf087baa3e3b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:46.245251 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] RBAC: Authorizing `identity:delete_user(user_id=95cd3336f6354d3bbe7aaf087baa3e3b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:46.247165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:46.302657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95cd3336f6354d3bbe7aaf087baa3e3b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.303305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95cd3336f6354d3bbe7aaf087baa3e3b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.304458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:46.324093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95cd3336f6354d3bbe7aaf087baa3e3b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.324876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95cd3336f6354d3bbe7aaf087baa3e3b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.332624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95cd3336f6354d3bbe7aaf087baa3e3b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.347945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 913/1837] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:13:46 2022] DELETE /identity/v3/users/95cd3336f6354d3bbe7aaf087baa3e3b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:46.352615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:46.352904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:46.353211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:46.370573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:46.371766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:46.432825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 925/1838] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:13:46.443036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:46.444528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:46.504046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f21740-35f5-430b-8658-a9b2fb40b09a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:46.507979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9eff24-8246-43f7-819e-3b58772b5a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:46.508636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eff24-8246-43f7-819e-3b58772b5a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:46.508915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eff24-8246-43f7-819e-3b58772b5a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:46.509217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eff24-8246-43f7-819e-3b58772b5a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:46.519107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9eff24-8246-43f7-819e-3b58772b5a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:46.520586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9eff24-8246-43f7-819e-3b58772b5a83 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:46.530171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 914/1839] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:46.809571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:46.811009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:46.823169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a01dbd-d46f-4e7d-9b11-cd6446c7732f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:46.828486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:46.829582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:46.829979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:46.830445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] PATH_INFO: `/v3/projects/7c646f4279764cc58afaf18685851707` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:46.832005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c646f4279764cc58afaf18685851707)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:46.834254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:46.886451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c646f4279764cc58afaf18685851707', 'request_id': 'req-603c82b7-d666-4d8b-a474-7372912fad95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.893092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c646f4279764cc58afaf18685851707', 'request_id': 'req-603c82b7-d666-4d8b-a474-7372912fad95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.898765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c646f4279764cc58afaf18685851707', 'request_id': 'req-603c82b7-d666-4d8b-a474-7372912fad95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.911667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] The token cache is being invalidate because project 7c646f4279764cc58afaf18685851707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:13:46.911956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.912575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-603c82b7-d666-4d8b-a474-7372912fad95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:13:46.914176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 926/1840] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:13:46 2022] DELETE /identity/v3/projects/7c646f4279764cc58afaf18685851707 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:13:46.919337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a905-25b5-4f61-a164-477115cefc03 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:46.919620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a905-25b5-4f61-a164-477115cefc03 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:46.919889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71a905-25b5-4f61-a164-477115cefc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:46.941667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8e71a905-25b5-4f61-a164-477115cefc03 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:46.942898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8e71a905-25b5-4f61-a164-477115cefc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:47.009599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 915/1841] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:47.018540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db896c8-0b62-46c3-b631-209fbf86177c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.018831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db896c8-0b62-46c3-b631-209fbf86177c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.019116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db896c8-0b62-46c3-b631-209fbf86177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.038776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3db896c8-0b62-46c3-b631-209fbf86177c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:47.039994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3db896c8-0b62-46c3-b631-209fbf86177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:47.048415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 927/1842] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:47.053830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a905-25b5-4f61-a164-477115cefc03 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:47.064322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71a905-25b5-4f61-a164-477115cefc03 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:47.069536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:47.070212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.070495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.070765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.071498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:13:47.071801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:47.073458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:47.089833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 916/1843] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:13:47.091230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db896c8-0b62-46c3-b631-209fbf86177c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:47.102425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db896c8-0b62-46c3-b631-209fbf86177c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:47.107221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:47.107888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.108151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.108412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.109142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:47.111034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:47.142579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 928/1844] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:13:47 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:47.146629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:47.156758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfd7f9-20cc-430c-8b81-675ab2ff5dc6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:47.207787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:47.215063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:47.215748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.216033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.216319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.217457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:47.219551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:47.241070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 917/1845] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:13:47 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:13:47.245369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:47.256274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462da30-f1cb-49ff-8a89-ff9c038c66e9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:47.261212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:47.261891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.262167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.262440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.263170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:47.264792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:47.275481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 929/1846] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:47.279586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:47.289911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c72628-e205-443c-8873-b5fef170421e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:47.294948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdd9ef-ddeb-4630-8e46-d470b3022c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:47.295976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd9ef-ddeb-4630-8e46-d470b3022c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.296286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd9ef-ddeb-4630-8e46-d470b3022c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.296519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd9ef-ddeb-4630-8e46-d470b3022c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.297281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdd9ef-ddeb-4630-8e46-d470b3022c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:47.298901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdd9ef-ddeb-4630-8e46-d470b3022c7c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:47.314304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 918/1847] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:13:47.315848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:47.326362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0de001-8988-403c-82d6-454dd396d199 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:47.332022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:47.333318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.333655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.333998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] PATH_INFO: `/v3/projects/2135ffc5da1943c4812e94e771f1b574/users/76d7878ef15a4a03b109cead2ac18861/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.346252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:47.351272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2135ffc5da1943c4812e94e771f1b574, user_id=76d7878ef15a4a03b109cead2ac18861, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:47.355221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:47.370488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 930/1848] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:13:47 2022] PUT /identity/v3/projects/2135ffc5da1943c4812e94e771f1b574/users/76d7878ef15a4a03b109cead2ac18861/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:13:47.380833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa290-9d5d-49dc-b184-6061ae9c4335 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.381154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa290-9d5d-49dc-b184-6061ae9c4335 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.381440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa290-9d5d-49dc-b184-6061ae9c4335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.409279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3a4fa290-9d5d-49dc-b184-6061ae9c4335 None None] MFA Rules not processed for user `76d7878ef15a4a03b109cead2ac18861`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:47.410534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3a4fa290-9d5d-49dc-b184-6061ae9c4335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:47.479752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 919/1849] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:13:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:47.491449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:47.493297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:47.558405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a401a0-abbe-4da4-a962-46abf10ce233 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:47.562153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13b597-48a7-4c75-be9c-01e0739e02b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:47.562754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b597-48a7-4c75-be9c-01e0739e02b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:47.563039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b597-48a7-4c75-be9c-01e0739e02b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:47.563318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b597-48a7-4c75-be9c-01e0739e02b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:47.572499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13b597-48a7-4c75-be9c-01e0739e02b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:47.573968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13b597-48a7-4c75-be9c-01e0739e02b4 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:47.583630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 931/1850] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:48.379724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a9db-682c-4720-b7a5-ee061b75f396 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:48.379724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a9db-682c-4720-b7a5-ee061b75f396 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:48.379724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a9db-682c-4720-b7a5-ee061b75f396 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:48.381209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 920/1851] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:13:48.393770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:48.394888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:48.395581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:48.428545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:13:48.431821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:48.539224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 932/1852] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:13:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:13:48.766342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03a9db-682c-4720-b7a5-ee061b75f396 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:48.768697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2e03a9db-682c-4720-b7a5-ee061b75f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:48.793116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03a9db-682c-4720-b7a5-ee061b75f396 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:48.802811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:48.803691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:48.805290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:48.805986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:48.809039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:48.986927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:48.995288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:49.028139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 921/1853] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:49.360413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:49.362183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:49.381526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870b52b-3bd3-4216-a758-95d383269aa0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:49.400282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:49.400857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:49.401895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:49.401895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:49.418249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:49.421832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:49.445226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 933/1854] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:13:59.020161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:59.433290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:13:59.447139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:13:59.457709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ec8d-2225-4549-a14d-8ddbe66fdf3b service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:13:59.462522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea7e1-c59d-48bd-a9cc-a71e2068884d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:13:59.463206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea7e1-c59d-48bd-a9cc-a71e2068884d service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:13:59.463485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea7e1-c59d-48bd-a9cc-a71e2068884d service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:13:59.463763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea7e1-c59d-48bd-a9cc-a71e2068884d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:13:59.465756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-55bea7e1-c59d-48bd-a9cc-a71e2068884d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:13:59.529296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea7e1-c59d-48bd-a9cc-a71e2068884d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:13:59.530845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea7e1-c59d-48bd-a9cc-a71e2068884d service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:13:59.540523 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 922/1855] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:13:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:07.159172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:07.160805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:07.232414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb03fe-5502-4acc-997d-e1e83ecd6911 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:07.238949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:07.240737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:07.241306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:07.241593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] PATH_INFO: `/v3/users/3a4bd68aa038431db58561da144e5072` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:07.242887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a4bd68aa038431db58561da144e5072)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:07.245002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:07.333599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a4bd68aa038431db58561da144e5072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:07.334972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a4bd68aa038431db58561da144e5072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:07.335714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a4bd68aa038431db58561da144e5072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:07.353390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a4bd68aa038431db58561da144e5072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:07.361030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a4bd68aa038431db58561da144e5072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:07.381793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 934/1856] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:07 2022] DELETE /identity/v3/users/3a4bd68aa038431db58561da144e5072 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:07.386258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:07.386804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:07.386804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:07.405860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:07.407062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:07.472420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 923/1857] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:07.482288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:07.483792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:07.559596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea19227-e809-4b68-a397-3de6da8af21d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:07.564718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:07.565579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:07.566089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:07.566613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:07.579440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:07.581014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:07.609346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 935/1858] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:07.914946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:07.916684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:07.934257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045cdfb-4b38-4a2c-b932-10daed3e05be None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:07.939449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:07.940389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:07.940674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:07.940981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] PATH_INFO: `/v3/projects/62028edb09ac4a50987b17d79b728af9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:07.942409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62028edb09ac4a50987b17d79b728af9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:07.944366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:08.006295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62028edb09ac4a50987b17d79b728af9', 'request_id': 'req-1cced819-2898-414d-b572-2382a50a66b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.014718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62028edb09ac4a50987b17d79b728af9', 'request_id': 'req-1cced819-2898-414d-b572-2382a50a66b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.022862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62028edb09ac4a50987b17d79b728af9', 'request_id': 'req-1cced819-2898-414d-b572-2382a50a66b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.033721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] The token cache is being invalidate because project 62028edb09ac4a50987b17d79b728af9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:08.034223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.035579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.037468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 924/1859] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:07 2022] DELETE /identity/v3/projects/62028edb09ac4a50987b17d79b728af9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:08.041916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:08.053269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e5c8e-8ac8-4958-bb91-fb3ec37c9b0a service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:08.073198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:08.115101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:08.116001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:08.116279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:08.116552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] PATH_INFO: `/v3/users/f96cb5a040d3486e948a8f2a6ff85a25` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:08.117767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96cb5a040d3486e948a8f2a6ff85a25)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:08.119732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:08.168410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96cb5a040d3486e948a8f2a6ff85a25'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.169089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96cb5a040d3486e948a8f2a6ff85a25'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.170090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96cb5a040d3486e948a8f2a6ff85a25'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.183277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96cb5a040d3486e948a8f2a6ff85a25'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.190981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96cb5a040d3486e948a8f2a6ff85a25'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.205304 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 936/1860] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:08 2022] DELETE /identity/v3/users/f96cb5a040d3486e948a8f2a6ff85a25 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:08.623709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:08.626427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:08.645561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cced819-2898-414d-b572-2382a50a66b3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:08.667979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:08.711595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:08.712485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:08.712765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:08.713042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] PATH_INFO: `/v3/projects/f5a54dd42c53452db63a8f8ac5dd45a3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:08.714279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a54dd42c53452db63a8f8ac5dd45a3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:08.716177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:08.768362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a54dd42c53452db63a8f8ac5dd45a3', 'request_id': 'req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.778111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a54dd42c53452db63a8f8ac5dd45a3', 'request_id': 'req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.785488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a54dd42c53452db63a8f8ac5dd45a3', 'request_id': 'req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.797615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] The token cache is being invalidate because project f5a54dd42c53452db63a8f8ac5dd45a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:08.797902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.798546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.800223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 925/1861] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:08 2022] DELETE /identity/v3/projects/f5a54dd42c53452db63a8f8ac5dd45a3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:08.804550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:08.806070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:08.819378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843d97-46f0-4a1c-b8e4-efa99d3931b2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:08.890913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:08.891813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:08.892095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:08.892390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] PATH_INFO: `/v3/users/a353a9ecf9ea4ebb97817a2269fd1f82` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:08.893614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a353a9ecf9ea4ebb97817a2269fd1f82)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:08.895929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:08.945826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:08.947952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a353a9ecf9ea4ebb97817a2269fd1f82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.948897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a353a9ecf9ea4ebb97817a2269fd1f82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.949943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a353a9ecf9ea4ebb97817a2269fd1f82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.963654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a353a9ecf9ea4ebb97817a2269fd1f82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.970836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e35ca40-9806-4f9e-b73f-be88218cde3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a353a9ecf9ea4ebb97817a2269fd1f82'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:08.987374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 937/1862] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:08 2022] DELETE /identity/v3/users/a353a9ecf9ea4ebb97817a2269fd1f82 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:09.271655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:09.273171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:09.286488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ff3a9-ccf1-4a9c-a345-c33c3f912235 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:09.346739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:09.347642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.347919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.348188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] PATH_INFO: `/v3/projects/9bd1eafb9b5c470aa77945302ae459de` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.349403 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd1eafb9b5c470aa77945302ae459de)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:09.351308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:09.398209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd1eafb9b5c470aa77945302ae459de', 'request_id': 'req-31269e8e-c226-4b07-b6b1-ac2e4399c6da'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:09.401977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:09.407090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd1eafb9b5c470aa77945302ae459de', 'request_id': 'req-31269e8e-c226-4b07-b6b1-ac2e4399c6da'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:09.414313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd1eafb9b5c470aa77945302ae459de', 'request_id': 'req-31269e8e-c226-4b07-b6b1-ac2e4399c6da'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:09.425743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] The token cache is being invalidate because project 9bd1eafb9b5c470aa77945302ae459de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:09.426030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:09.426654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-31269e8e-c226-4b07-b6b1-ac2e4399c6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:09.428215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 926/1863] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:09 2022] DELETE /identity/v3/projects/9bd1eafb9b5c470aa77945302ae459de => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:09.434326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34b562-0c62-4bf3-b349-7ddc52a65085 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.434826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34b562-0c62-4bf3-b349-7ddc52a65085 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.434944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34b562-0c62-4bf3-b349-7ddc52a65085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.453836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bd34b562-0c62-4bf3-b349-7ddc52a65085 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:09.455038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bd34b562-0c62-4bf3-b349-7ddc52a65085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:09.530021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 938/1864] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:09.540257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e3a6-b637-4478-849f-bb5584941140 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.540542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e3a6-b637-4478-849f-bb5584941140 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.540817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e3a6-b637-4478-849f-bb5584941140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.560149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7ef6e3a6-b637-4478-849f-bb5584941140 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:09.561374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7ef6e3a6-b637-4478-849f-bb5584941140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:09.573705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 927/1865] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:09.576071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34b562-0c62-4bf3-b349-7ddc52a65085 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:09.587167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34b562-0c62-4bf3-b349-7ddc52a65085 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:09.592050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:09.592724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.593001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.593318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.594045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:09.594348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:09.596080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:09.608403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 939/1866] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:09.612798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e3a6-b637-4478-849f-bb5584941140 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:09.621952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e3a6-b637-4478-849f-bb5584941140 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:09.626434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:09.627094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.627376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.627646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.628409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:09.630323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:09.660900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 928/1867] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:09 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:09.662476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:09.673474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197fc9-3035-460c-ab5d-a5d447bf85c7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:09.732937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:09.733686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.733965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.734234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.734935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:09.736885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:09.758496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 940/1868] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:09 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:09.762880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:09.773926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508a546-0143-44f5-b2c9-475f6e155e02 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:09.779321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:09.780006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.780362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.781344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.782092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:09.783700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:09.797107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 929/1869] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:09.800752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:09.815533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78b0fd-e527-4663-a720-4416689d0d98 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:09.820777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a828d0-3899-4d31-b294-4a26bf77691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:09.822164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a828d0-3899-4d31-b294-4a26bf77691c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.822164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a828d0-3899-4d31-b294-4a26bf77691c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.822164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a828d0-3899-4d31-b294-4a26bf77691c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.822891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a828d0-3899-4d31-b294-4a26bf77691c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:09.824583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a828d0-3899-4d31-b294-4a26bf77691c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:09.836911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 941/1870] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:09.841207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:09.851674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98f948-ada8-439e-b131-0982913730ba admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:09.856226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:09.857152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.857435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.857707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] PATH_INFO: `/v3/projects/14e59442bc59490b81fa477c29dd2a45/users/e60e852b035a47cc9a5978d93d9a8fce/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.873836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e59442bc59490b81fa477c29dd2a45, user_id=e60e852b035a47cc9a5978d93d9a8fce, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:09.876581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:09.895149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 930/1871] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:09 2022] PUT /identity/v3/projects/14e59442bc59490b81fa477c29dd2a45/users/e60e852b035a47cc9a5978d93d9a8fce/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:09.907349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635bb42-9901-4f4b-a922-a6e4f79934b0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:09.907648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635bb42-9901-4f4b-a922-a6e4f79934b0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:09.907914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635bb42-9901-4f4b-a922-a6e4f79934b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:09.935331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6635bb42-9901-4f4b-a922-a6e4f79934b0 None None] MFA Rules not processed for user `e60e852b035a47cc9a5978d93d9a8fce`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:09.936702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6635bb42-9901-4f4b-a922-a6e4f79934b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:09.951419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:10.014812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 942/1872] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:10.019704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.095931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c17bc2-cdf8-4913-9c3a-fb4910bd3174 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.100817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.101517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.101802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.102079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.102773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.104667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.134501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 931/1873] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:10 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:10.135689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635bb42-9901-4f4b-a922-a6e4f79934b0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.148081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635bb42-9901-4f4b-a922-a6e4f79934b0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.211142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.211830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.212107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.212377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.213086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.214980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.236294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 943/1874] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:10 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:10.241257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.254232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de235558-454b-419e-bc84-4527dd131ca8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.259388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.260339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.260545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.260886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.261708 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.263353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.274696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 932/1875] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:10.278894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.289076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b08882-0d64-4917-8a36-e9b62d77dd13 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.293812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.295096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.295350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.295620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] PATH_INFO: `/v3/projects/05a0807deb0d43ddbd9e7959f175e535/users/0eb9d879e5184a6ca30156c8dca4946b/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.310317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a0807deb0d43ddbd9e7959f175e535, user_id=0eb9d879e5184a6ca30156c8dca4946b, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.312902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.328573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 944/1876] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:10 2022] PUT /identity/v3/projects/05a0807deb0d43ddbd9e7959f175e535/users/0eb9d879e5184a6ca30156c8dca4946b/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:10.333183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.347386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d713596-3599-4753-bf65-6d1fff77a32d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.394857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:10.407287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6f98e-3285-4867-9a5c-34994fe24a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.407972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6f98e-3285-4867-9a5c-34994fe24a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.408286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6f98e-3285-4867-9a5c-34994fe24a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.408565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6f98e-3285-4867-9a5c-34994fe24a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.409633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6f98e-3285-4867-9a5c-34994fe24a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.412049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6f98e-3285-4867-9a5c-34994fe24a01 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.426110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 933/1877] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:10.430444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.442382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c87fd3-32b5-4112-8125-e2b2e9b1128d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.448421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.449363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.449643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.449913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] PATH_INFO: `/v3/projects/05a0807deb0d43ddbd9e7959f175e535/users/0eb9d879e5184a6ca30156c8dca4946b/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.452166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a0807deb0d43ddbd9e7959f175e535, user_id=0eb9d879e5184a6ca30156c8dca4946b, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.455004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.467273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 945/1878] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:10 2022] PUT /identity/v3/projects/05a0807deb0d43ddbd9e7959f175e535/users/0eb9d879e5184a6ca30156c8dca4946b/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:10.478751 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.479186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.479623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.521010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] MFA Rules not processed for user `0eb9d879e5184a6ca30156c8dca4946b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:10.522512 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:10.598580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 934/1879] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:10.609609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.611384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:10.673675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959876c-2430-4a5a-84bb-781c2f2d0c09 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.678729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.679422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.679707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.679985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.690061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.692027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.702175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 946/1880] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:10.748574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:10.750119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:10.762896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad21f1-7344-4df6-a435-9f6389ce4f1d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:10.768034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdf176-3499-4dcd-a66b-9860a380b4dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:10.769323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdf176-3499-4dcd-a66b-9860a380b4dc service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:10.769861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdf176-3499-4dcd-a66b-9860a380b4dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:10.770387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdf176-3499-4dcd-a66b-9860a380b4dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:10.843548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdf176-3499-4dcd-a66b-9860a380b4dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:10.845089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdf176-3499-4dcd-a66b-9860a380b4dc service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:10.855358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 935/1881] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:12.566000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:12.568231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:12.601800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:12.698330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fedfa-59b6-4013-ba9e-1de24a4ef768 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:12.705817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:12.706857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:12.707493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:12.707858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] PATH_INFO: `/v3/users/76d7878ef15a4a03b109cead2ac18861` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:12.724501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d7878ef15a4a03b109cead2ac18861)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:12.727128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:12.823081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d7878ef15a4a03b109cead2ac18861'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:12.824623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d7878ef15a4a03b109cead2ac18861'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:12.825641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d7878ef15a4a03b109cead2ac18861'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:12.881577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d7878ef15a4a03b109cead2ac18861'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:12.896612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d7878ef15a4a03b109cead2ac18861'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:12.931730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 947/1882] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:12 2022] DELETE /identity/v3/users/76d7878ef15a4a03b109cead2ac18861 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:12.933607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:12.934093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:12.934187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:12.955553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:12.958020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:13.048990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 936/1883] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:13.062264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:13.064250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:13.145423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3634a05-2d3f-45e4-a888-f6dcf21db7d4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:13.150786 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e243e3-9aae-4caa-aa2c-a0fcc0e67adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:13.151574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e243e3-9aae-4caa-aa2c-a0fcc0e67adb service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:13.151860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e243e3-9aae-4caa-aa2c-a0fcc0e67adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:13.152988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e243e3-9aae-4caa-aa2c-a0fcc0e67adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:13.166164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e243e3-9aae-4caa-aa2c-a0fcc0e67adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:13.168111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e243e3-9aae-4caa-aa2c-a0fcc0e67adb service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:13.182627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 948/1884] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:14.155923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:14.157475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:14.182265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c759c-f0a7-4bf9-8228-8a60c1066aee None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:14.189232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:14.190067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.191051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.191519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] PATH_INFO: `/v3/projects/2135ffc5da1943c4812e94e771f1b574` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.192935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2135ffc5da1943c4812e94e771f1b574)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:14.194960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:14.203365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:14.258360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2135ffc5da1943c4812e94e771f1b574', 'request_id': 'req-917ef73b-a32e-4764-8630-9e2d1f037848'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:14.270344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2135ffc5da1943c4812e94e771f1b574', 'request_id': 'req-917ef73b-a32e-4764-8630-9e2d1f037848'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:14.279384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2135ffc5da1943c4812e94e771f1b574', 'request_id': 'req-917ef73b-a32e-4764-8630-9e2d1f037848'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:14.292593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] The token cache is being invalidate because project 2135ffc5da1943c4812e94e771f1b574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:14.293004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:14.293782 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-917ef73b-a32e-4764-8630-9e2d1f037848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:14.299337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 937/1885] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:14 2022] DELETE /identity/v3/projects/2135ffc5da1943c4812e94e771f1b574 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:14.302305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b3b2-2cd2-4867-820d-b6c9a1e571ec None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.302788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b3b2-2cd2-4867-820d-b6c9a1e571ec None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.303050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b3b2-2cd2-4867-820d-b6c9a1e571ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.327504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6103b3b2-2cd2-4867-820d-b6c9a1e571ec None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:14.328666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6103b3b2-2cd2-4867-820d-b6c9a1e571ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:14.367366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:14.407038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 949/1886] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:14.414938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a1c2c-f7ce-4b74-b0c4-a3c4861ed603 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.415337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a1c2c-f7ce-4b74-b0c4-a3c4861ed603 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.415687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a1c2c-f7ce-4b74-b0c4-a3c4861ed603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.447823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ba2a1c2c-f7ce-4b74-b0c4-a3c4861ed603 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:14.449355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba2a1c2c-f7ce-4b74-b0c4-a3c4861ed603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:14.463670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 938/1887] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:14.468176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b3b2-2cd2-4867-820d-b6c9a1e571ec None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:14.484804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b3b2-2cd2-4867-820d-b6c9a1e571ec None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:14.499859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:14.508437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.508437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.508437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.508437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:14.508437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:14.509225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:14.532250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 950/1888] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:14.533494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a1c2c-f7ce-4b74-b0c4-a3c4861ed603 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:14.545080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a1c2c-f7ce-4b74-b0c4-a3c4861ed603 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:14.556357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:14.557483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.557848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.558132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.558925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:14.561821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:14.608502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 939/1889] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:14 2022] POST /identity/v3/projects => generated 399 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:14.610083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:14.631065 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55796f3-d37b-4b42-b908-4743b0d2c160 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:14.718425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:14.719238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.719620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.719991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.720928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:14.723726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:14.753969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 951/1890] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:14 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:14.758064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:14.768827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79521a6d-b2ac-4f03-b477-6a1d0a6d6874 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:14.773791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:14.774475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.774747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.775017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.775749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:14.777385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:14.792002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 940/1891] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:14.793278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:14.804187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db93eb-1dd2-4538-85ed-eefd150717bf admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:14.812147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef6a77-847f-46d7-81f5-3de8723fbb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:14.812838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef6a77-847f-46d7-81f5-3de8723fbb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.813400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef6a77-847f-46d7-81f5-3de8723fbb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.813543 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef6a77-847f-46d7-81f5-3de8723fbb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.814585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef6a77-847f-46d7-81f5-3de8723fbb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:14.816854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef6a77-847f-46d7-81f5-3de8723fbb07 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:14.827446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 952/1892] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:14.831580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:14.843132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242fb63-9812-464c-be5e-3eb8d799b13d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:14.848754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:14.849757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.850096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.850379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] PATH_INFO: `/v3/projects/a28d2292fb4d4e67a9baa599bc4c05f0/users/ef2b86c3ebe04b65b81a8434e9709bb9/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.878182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28d2292fb4d4e67a9baa599bc4c05f0, user_id=ef2b86c3ebe04b65b81a8434e9709bb9, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:14.881345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:14.897429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 941/1893] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:14 2022] PUT /identity/v3/projects/a28d2292fb4d4e67a9baa599bc4c05f0/users/ef2b86c3ebe04b65b81a8434e9709bb9/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:14.908595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:14.908994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:14.909632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:14.938500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] MFA Rules not processed for user `ef2b86c3ebe04b65b81a8434e9709bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:14.939977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:15.020704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 953/1894] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:15.032257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:15.034558 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:15.096824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c587d2a-9c90-47ef-9626-61a97a4bb16e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:15.101111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85cacc-9307-407e-89e3-757243ba000f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:15.101776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cacc-9307-407e-89e3-757243ba000f service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:15.102059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cacc-9307-407e-89e3-757243ba000f service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:15.102334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cacc-9307-407e-89e3-757243ba000f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:15.111054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85cacc-9307-407e-89e3-757243ba000f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:15.112609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85cacc-9307-407e-89e3-757243ba000f service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:15.121989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 942/1895] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:14:17.852651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:17.858595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:17.896640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:17.932020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6127da-91fa-4c62-b6ed-545ee29a54f1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:17.936554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:17.937456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:17.937739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:17.938012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] PATH_INFO: `/v3/users/e60e852b035a47cc9a5978d93d9a8fce` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:17.939191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60e852b035a47cc9a5978d93d9a8fce)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:17.941130 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:17.989061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60e852b035a47cc9a5978d93d9a8fce'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:17.989815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60e852b035a47cc9a5978d93d9a8fce'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:17.990420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60e852b035a47cc9a5978d93d9a8fce'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.003731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60e852b035a47cc9a5978d93d9a8fce'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.010829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60e852b035a47cc9a5978d93d9a8fce'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.028356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 954/1896] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:17 2022] DELETE /identity/v3/users/e60e852b035a47cc9a5978d93d9a8fce => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:18.032850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:18.033304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:18.033447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:18.052001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:18.053350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:18.084676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:18.116211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 943/1897] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:18.127712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:18.129264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:18.191100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954481e-9468-4ad2-b5ec-7da4befe705e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:18.194772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:18.195445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:18.195750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:18.196017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:18.205078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:18.206578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:18.216247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 955/1898] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:18.508546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:18.510122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:18.522616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a1198-1e9f-4701-92a3-bf7ef7433ee8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:18.528125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:18.529096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:18.529423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:18.529804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] PATH_INFO: `/v3/projects/14e59442bc59490b81fa477c29dd2a45` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:18.531067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14e59442bc59490b81fa477c29dd2a45)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:18.532982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:18.583624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14e59442bc59490b81fa477c29dd2a45', 'request_id': 'req-14c95695-e0ba-44e8-8212-94d5e0d76638'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.592487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14e59442bc59490b81fa477c29dd2a45', 'request_id': 'req-14c95695-e0ba-44e8-8212-94d5e0d76638'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.599585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14e59442bc59490b81fa477c29dd2a45', 'request_id': 'req-14c95695-e0ba-44e8-8212-94d5e0d76638'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.611531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] The token cache is being invalidate because project 14e59442bc59490b81fa477c29dd2a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:18.611825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.612505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.614109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 944/1899] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:18 2022] DELETE /identity/v3/projects/14e59442bc59490b81fa477c29dd2a45 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:18.619269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:18.628861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc346f7-b638-4009-8f35-eafc6ef4b127 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:18.688322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:18.701869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:18.702806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:18.703051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:18.703325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] PATH_INFO: `/v3/users/0eb9d879e5184a6ca30156c8dca4946b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:18.704548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb9d879e5184a6ca30156c8dca4946b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:18.706753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:18.757899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb9d879e5184a6ca30156c8dca4946b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.758576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb9d879e5184a6ca30156c8dca4946b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.759258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb9d879e5184a6ca30156c8dca4946b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.774467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb9d879e5184a6ca30156c8dca4946b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.779499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2e35c228-3283-4932-95b5-c9814500c83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb9d879e5184a6ca30156c8dca4946b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:18.794787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 956/1900] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:18 2022] DELETE /identity/v3/users/0eb9d879e5184a6ca30156c8dca4946b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:19.149473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.151127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:19.165344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95695-e0ba-44e8-8212-94d5e0d76638 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.225141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.226120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.226401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.226667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] PATH_INFO: `/v3/projects/05a0807deb0d43ddbd9e7959f175e535` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.227882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a0807deb0d43ddbd9e7959f175e535)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.229865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:19.278228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a0807deb0d43ddbd9e7959f175e535', 'request_id': 'req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:19.287538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a0807deb0d43ddbd9e7959f175e535', 'request_id': 'req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:19.294955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a0807deb0d43ddbd9e7959f175e535', 'request_id': 'req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:19.296317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:19.308037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] The token cache is being invalidate because project 05a0807deb0d43ddbd9e7959f175e535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:19.308329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:19.309223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d6b90fa3-5760-4b4a-bad6-58e7dea2e997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:19.314802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 945/1901] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:19 2022] DELETE /identity/v3/projects/05a0807deb0d43ddbd9e7959f175e535 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:19.316569 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dadc6-b6f8-4e46-b57b-dc8f405fda86 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.317039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dadc6-b6f8-4e46-b57b-dc8f405fda86 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.317180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dadc6-b6f8-4e46-b57b-dc8f405fda86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.335026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f24dadc6-b6f8-4e46-b57b-dc8f405fda86 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:19.336210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f24dadc6-b6f8-4e46-b57b-dc8f405fda86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:19.397820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 957/1902] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:19.406942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad7603-f053-4459-9fac-fbe277a29b42 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.407230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad7603-f053-4459-9fac-fbe277a29b42 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.407509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad7603-f053-4459-9fac-fbe277a29b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.427644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-48ad7603-f053-4459-9fac-fbe277a29b42 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:19.428896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-48ad7603-f053-4459-9fac-fbe277a29b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:19.437644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 946/1903] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:19.441825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24dadc6-b6f8-4e46-b57b-dc8f405fda86 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.452456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24dadc6-b6f8-4e46-b57b-dc8f405fda86 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.457847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.458525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.458841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.459110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.460038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:19.460332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.462005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:19.474275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 958/1904] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:19.478445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad7603-f053-4459-9fac-fbe277a29b42 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.487666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad7603-f053-4459-9fac-fbe277a29b42 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.492540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.493221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.493509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.493818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.494506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.496394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:19.524836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 947/1905] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:19 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:19.529197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.539767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba4a6a-bb8f-4f3c-b53e-f7cefadf7fae admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.599921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:19.602516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.603251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.603519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.603838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.604681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.607520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:19.638424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 959/1906] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:19 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:19.640250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.654214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecf999-22aa-48e6-a348-b3d0321d8719 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.659869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.660539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.660822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.661126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.661869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.663517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:19.674953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 948/1907] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:19.679355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.690537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902c77b-46fa-49a5-99c2-36f559e34810 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.697134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217b927-d5c9-4974-8687-cffe7a13fb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.697798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217b927-d5c9-4974-8687-cffe7a13fb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.698144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217b927-d5c9-4974-8687-cffe7a13fb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.698420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217b927-d5c9-4974-8687-cffe7a13fb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.699232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217b927-d5c9-4974-8687-cffe7a13fb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.701211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217b927-d5c9-4974-8687-cffe7a13fb3f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:19.713305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 960/1908] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:19.722717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.738562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb1dea-ffc1-463a-af3b-c84fd81087e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.744222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.745120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.745405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.745675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] PATH_INFO: `/v3/projects/11e231d1dfc6420994c01f312bd5221f/users/9886a7a4b7044bd18e6b1cc5d1407cca/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.762287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e231d1dfc6420994c01f312bd5221f, user_id=9886a7a4b7044bd18e6b1cc5d1407cca, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.764969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:19.781532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 949/1909] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:19 2022] PUT /identity/v3/projects/11e231d1dfc6420994c01f312bd5221f/users/9886a7a4b7044bd18e6b1cc5d1407cca/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:19.797097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeed5c5-c364-4f98-b5e1-0e35230e7d63 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.797424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeed5c5-c364-4f98-b5e1-0e35230e7d63 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.797884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeed5c5-c364-4f98-b5e1-0e35230e7d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.830898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cdeed5c5-c364-4f98-b5e1-0e35230e7d63 None None] MFA Rules not processed for user `9886a7a4b7044bd18e6b1cc5d1407cca`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:19.832123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cdeed5c5-c364-4f98-b5e1-0e35230e7d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:19.910705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 961/1910] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:19.916185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:19.989399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536e0e-557e-46d0-a75a-5a9de3eca0e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:19.994433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:19.995170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:19.995429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:19.996064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:19.996430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:19.998420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.029026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 950/1911] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:19 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:20.033440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeed5c5-c364-4f98-b5e1-0e35230e7d63 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:20.045712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeed5c5-c364-4f98-b5e1-0e35230e7d63 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:20.114005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:20.114694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.114998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.115273 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.116175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:20.118368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.143870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 962/1912] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:20 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:20.147971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:20.159779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7db85-c0d0-423f-aa9a-5ddd963a7e94 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:20.165082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:20.165728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.166003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.166281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.167020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:20.168660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.181581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 951/1913] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:20.186407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:20.200476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44155fac-f38a-4b60-b80e-e8f3335777c8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:20.205949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:20.208214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.208214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.208214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] PATH_INFO: `/v3/projects/a3cbd86a677049c085024cdb7881c4e7/users/9f781ab464934c228ba7da7fe539894c/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.225889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cbd86a677049c085024cdb7881c4e7, user_id=9f781ab464934c228ba7da7fe539894c, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:20.233576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.251141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 963/1914] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:20 2022] PUT /identity/v3/projects/a3cbd86a677049c085024cdb7881c4e7/users/9f781ab464934c228ba7da7fe539894c/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:20.256380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:20.267944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0b1ce-d3eb-4150-9057-b1e20d693ec6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:20.334841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc118dd-4754-4543-a093-6d94fb314472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:20.335533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc118dd-4754-4543-a093-6d94fb314472 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.335823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc118dd-4754-4543-a093-6d94fb314472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.336085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc118dd-4754-4543-a093-6d94fb314472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.336817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc118dd-4754-4543-a093-6d94fb314472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:20.339121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc118dd-4754-4543-a093-6d94fb314472 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.340593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:20.352109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 952/1915] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:20.357765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:20.369937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4138a-f9d2-46e2-8b39-c3c720b3f979 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:20.374769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:20.377111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.377111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.377111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] PATH_INFO: `/v3/projects/a3cbd86a677049c085024cdb7881c4e7/users/9f781ab464934c228ba7da7fe539894c/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.378262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cbd86a677049c085024cdb7881c4e7, user_id=9f781ab464934c228ba7da7fe539894c, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:20.380909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.392707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 964/1916] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:20 2022] PUT /identity/v3/projects/a3cbd86a677049c085024cdb7881c4e7/users/9f781ab464934c228ba7da7fe539894c/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:20.404634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b063f-3aff-4ef9-b57c-10d18690fcfe None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.405169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b063f-3aff-4ef9-b57c-10d18690fcfe None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.405647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b063f-3aff-4ef9-b57c-10d18690fcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.435970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d98b063f-3aff-4ef9-b57c-10d18690fcfe None None] MFA Rules not processed for user `9f781ab464934c228ba7da7fe539894c`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:20.438473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d98b063f-3aff-4ef9-b57c-10d18690fcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:20.524925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 953/1917] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:20.542146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:20.543813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:20.566137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:20.607328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a0fc2-fb79-49f0-8070-563449917d63 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:20.611781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a69c9c-0ce8-4c40-824c-512fb0363720 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:20.612434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a69c9c-0ce8-4c40-824c-512fb0363720 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.612710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a69c9c-0ce8-4c40-824c-512fb0363720 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.614017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a69c9c-0ce8-4c40-824c-512fb0363720 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.625253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a69c9c-0ce8-4c40-824c-512fb0363720 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:20.626704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a69c9c-0ce8-4c40-824c-512fb0363720 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.641782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 965/1918] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:20.784443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b063f-3aff-4ef9-b57c-10d18690fcfe None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:20.849258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b063f-3aff-4ef9-b57c-10d18690fcfe None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:20.855773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:20.856663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.856946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.857248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] PATH_INFO: `/v3/users/9886a7a4b7044bd18e6b1cc5d1407cca` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.858472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9886a7a4b7044bd18e6b1cc5d1407cca)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:20.860387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:20.919711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9886a7a4b7044bd18e6b1cc5d1407cca'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:20.920492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9886a7a4b7044bd18e6b1cc5d1407cca'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:20.936019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9886a7a4b7044bd18e6b1cc5d1407cca'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:20.936718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9886a7a4b7044bd18e6b1cc5d1407cca'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:20.944056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9886a7a4b7044bd18e6b1cc5d1407cca'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:20.954960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:20.959881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 954/1919] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:20 2022] DELETE /identity/v3/users/9886a7a4b7044bd18e6b1cc5d1407cca => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:20.964825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:20.965196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:20.965514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:20.983278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:20.984473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:21.059120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 966/1920] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:21.069577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:21.071248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:21.142767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbffd2-74be-481a-9e38-f2970a90d3c0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:21.148624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:21.149308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:21.149586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:21.149866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:21.159845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:21.161295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:21.171243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 955/1921] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:21.483464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:21.485013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:21.505072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd35c00-6851-432e-bee6-be9b66b3bc0a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:21.510281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:21.511168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:21.511571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:21.511850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] PATH_INFO: `/v3/projects/11e231d1dfc6420994c01f312bd5221f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:21.513302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e231d1dfc6420994c01f312bd5221f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:21.515646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:21.570226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e231d1dfc6420994c01f312bd5221f', 'request_id': 'req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.576938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e231d1dfc6420994c01f312bd5221f', 'request_id': 'req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.584102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e231d1dfc6420994c01f312bd5221f', 'request_id': 'req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.612417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] The token cache is being invalidate because project 11e231d1dfc6420994c01f312bd5221f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:21.612775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.614435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.616589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 967/1922] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:21 2022] DELETE /identity/v3/projects/11e231d1dfc6420994c01f312bd5221f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:21.620637 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:21.633673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e5415-fd36-44fc-aa39-ef5f2c5cb0ca service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:21.698662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:21.699553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:21.699828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:21.700099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] PATH_INFO: `/v3/users/9f781ab464934c228ba7da7fe539894c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:21.701300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f781ab464934c228ba7da7fe539894c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:21.703212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:21.753851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f781ab464934c228ba7da7fe539894c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.754610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f781ab464934c228ba7da7fe539894c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.768422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f781ab464934c228ba7da7fe539894c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.769249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f781ab464934c228ba7da7fe539894c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.776594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-671a8ea4-36e0-47c9-8565-c9e69d602313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f781ab464934c228ba7da7fe539894c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:21.791049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 956/1923] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:21 2022] DELETE /identity/v3/users/9f781ab464934c228ba7da7fe539894c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:22.147423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.149035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:22.164067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7b2f5-81e0-403b-86e6-09ea746e20ba admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:22.209327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:22.232328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:22.233293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.233573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.233848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] PATH_INFO: `/v3/projects/a3cbd86a677049c085024cdb7881c4e7` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.235061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cbd86a677049c085024cdb7881c4e7)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:22.238110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:22.293582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cbd86a677049c085024cdb7881c4e7', 'request_id': 'req-5a6c05cc-06c8-4571-844a-3be846b19a3b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:22.300847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cbd86a677049c085024cdb7881c4e7', 'request_id': 'req-5a6c05cc-06c8-4571-844a-3be846b19a3b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:22.313610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cbd86a677049c085024cdb7881c4e7', 'request_id': 'req-5a6c05cc-06c8-4571-844a-3be846b19a3b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:22.329968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] The token cache is being invalidate because project a3cbd86a677049c085024cdb7881c4e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:22.330350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:22.331069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5a6c05cc-06c8-4571-844a-3be846b19a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:22.333249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 968/1924] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:22 2022] DELETE /identity/v3/projects/a3cbd86a677049c085024cdb7881c4e7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:22.338275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31429c0-6f76-4594-849e-069605aaedda None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.338760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31429c0-6f76-4594-849e-069605aaedda None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.338847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31429c0-6f76-4594-849e-069605aaedda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.357590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f31429c0-6f76-4594-849e-069605aaedda None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:22.359047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f31429c0-6f76-4594-849e-069605aaedda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:22.377464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:22.423681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 957/1925] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:22.433061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328734c4-febb-4c87-8415-9a995eee72f1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.434186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328734c4-febb-4c87-8415-9a995eee72f1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.434186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328734c4-febb-4c87-8415-9a995eee72f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.452924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-328734c4-febb-4c87-8415-9a995eee72f1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:22.454288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-328734c4-febb-4c87-8415-9a995eee72f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:22.464295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 969/1926] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:22.469270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31429c0-6f76-4594-849e-069605aaedda None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.479950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31429c0-6f76-4594-849e-069605aaedda None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:22.485120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:22.485793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.486070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.486341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.487129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:22.487429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:22.489526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:22.506717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 958/1927] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:22.507975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328734c4-febb-4c87-8415-9a995eee72f1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.517972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328734c4-febb-4c87-8415-9a995eee72f1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:22.522652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:22.523330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.523610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.523883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.524583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:22.526552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:22.565557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 970/1928] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:22 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:22.566789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.577829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58db4d9-f0a5-45f8-ab49-d5380a13d612 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:22.666385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:22.667353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.668545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.669410 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.670325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:22.674846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:22.704713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 959/1929] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:22 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:22.706167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.718250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75d9b-4635-493b-9c5a-a7642cd7d4d7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:22.723795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:22.724512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.724832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.725229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.726265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:22.727931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:22.740267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 971/1930] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:22.744682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.755066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7b91-5c77-479f-8967-1b635e9ab8ad admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:22.760938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec86b8b-a204-479a-9e1a-3e25923a66ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:22.761684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec86b8b-a204-479a-9e1a-3e25923a66ab admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.761934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec86b8b-a204-479a-9e1a-3e25923a66ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.762191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec86b8b-a204-479a-9e1a-3e25923a66ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.762910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec86b8b-a204-479a-9e1a-3e25923a66ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:22.764501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec86b8b-a204-479a-9e1a-3e25923a66ab admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:22.777109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 960/1931] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:22.780553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.791303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8d0e-e022-4cbd-b105-4796642aa3a5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:22.796463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:22.797541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.797880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.798231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] PATH_INFO: `/v3/projects/f709746f23924361870ad687842f0992/users/8e511b1648b148bc8e3bc2ebb8f6ea6b/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.814638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f709746f23924361870ad687842f0992, user_id=8e511b1648b148bc8e3bc2ebb8f6ea6b, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:22.817757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:22.833018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 972/1932] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:22 2022] PUT /identity/v3/projects/f709746f23924361870ad687842f0992/users/8e511b1648b148bc8e3bc2ebb8f6ea6b/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:22.843592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:22.843886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:22.844164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:22.872570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] MFA Rules not processed for user `8e511b1648b148bc8e3bc2ebb8f6ea6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:22.873860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:22.938060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 961/1933] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:22.949433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.951187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:22.993530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:22.995121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.019619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:23.019821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89106d4-525a-4ff5-a11c-7caabcd1e2ba admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:23.024102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:23.024849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.025232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.025579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.034641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:23.036160 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:23.062316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 973/1934] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:23.072073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:23.078236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc9c25-ae07-4807-aa8d-41ba91c2fd37 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:23.083193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fb877-abcb-4370-b103-e80431fd9205 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:23.084540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:23.085751 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.086079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.086362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] PATH_INFO: `/v3/users/be595828d401488d9390f584be914376` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.087673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] RBAC: Authorizing `identity:delete_user(user_id=be595828d401488d9390f584be914376)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:23.089162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:23.089797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:23.090055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.090341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.090620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] PATH_INFO: `/v3/users/8e511b1648b148bc8e3bc2ebb8f6ea6b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.091832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e511b1648b148bc8e3bc2ebb8f6ea6b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:23.093793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:23.101198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:23.151904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be595828d401488d9390f584be914376'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.152395 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e511b1648b148bc8e3bc2ebb8f6ea6b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.152922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be595828d401488d9390f584be914376'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.153161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e511b1648b148bc8e3bc2ebb8f6ea6b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.153794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e511b1648b148bc8e3bc2ebb8f6ea6b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.167882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e511b1648b148bc8e3bc2ebb8f6ea6b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.168721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be595828d401488d9390f584be914376'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.170149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be595828d401488d9390f584be914376'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.176077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3219b538-75fe-458a-ac68-dc2dd67a09fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e511b1648b148bc8e3bc2ebb8f6ea6b'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.179617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f3f1fab-b340-4941-9b24-4dc56694941d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be595828d401488d9390f584be914376'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.194952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 974/1935] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:23 2022] DELETE /identity/v3/users/8e511b1648b148bc8e3bc2ebb8f6ea6b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:23.199394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 962/1936] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:22 2022] DELETE /identity/v3/users/be595828d401488d9390f584be914376 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:23.201846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.202063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.203560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.204100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.205600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.205879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.226340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:23.227873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.232255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:23.233584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.305633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 975/1937] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:23.315977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:23.317542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.324487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 963/1938] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:23.340003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:23.340003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.376137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692d174-d1ca-44ad-b4e6-60165dc8531d None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:23.379736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:23.380389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.380667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.380995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.382829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:23.390688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:23.392264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:23.402393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e53a86-8076-45ca-84b8-dd351378125f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:23.403409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 976/1939] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:14:23.406340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:23.407023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.407301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.407543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.417436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:23.419236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:23.429688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 964/1940] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:14:23.798436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:23.800773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.815785 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63044b4d-96b7-449e-a89f-5544e8f7ccf1 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:23.822236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:23.823335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.823683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.824024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] PATH_INFO: `/v3/projects/f709746f23924361870ad687842f0992` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.825459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f709746f23924361870ad687842f0992)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:23.827763 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:23.857274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:23.873440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:23.875563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.903256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe938-c238-4338-a25b-086557323e7c service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:23.909481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f709746f23924361870ad687842f0992', 'request_id': 'req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.917096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f709746f23924361870ad687842f0992', 'request_id': 'req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.924865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f709746f23924361870ad687842f0992', 'request_id': 'req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.940956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] The token cache is being invalidate because project f709746f23924361870ad687842f0992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:23.941384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.942082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4a90a5e0-5e01-4496-b307-e7b3a317ab8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:23.944614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 977/1941] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:23 2022] DELETE /identity/v3/projects/f709746f23924361870ad687842f0992 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:23.949530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695ab5c-ef57-46dc-80a7-a953f95c5ce0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.949931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695ab5c-ef57-46dc-80a7-a953f95c5ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.950308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695ab5c-ef57-46dc-80a7-a953f95c5ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.970228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4695ab5c-ef57-46dc-80a7-a953f95c5ce0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:23.971592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4695ab5c-ef57-46dc-80a7-a953f95c5ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:23.973845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:23.975058 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.975344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.975979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] PATH_INFO: `/v3/projects/1bffe8e53f534c1ba596feccd9de7acb` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:23.977770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bffe8e53f534c1ba596feccd9de7acb)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:23.981264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 978/1942] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:23.981833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:23.989133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1831d2-b0f8-4ca7-a31a-cba52a909821 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:23.989520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1831d2-b0f8-4ca7-a31a-cba52a909821 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:23.989898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1831d2-b0f8-4ca7-a31a-cba52a909821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.010737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4a1831d2-b0f8-4ca7-a31a-cba52a909821 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:24.012024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4a1831d2-b0f8-4ca7-a31a-cba52a909821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.020962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 979/1943] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:24.026575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1831d2-b0f8-4ca7-a31a-cba52a909821 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.036982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1831d2-b0f8-4ca7-a31a-cba52a909821 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.039259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bffe8e53f534c1ba596feccd9de7acb', 'request_id': 'req-39484f6d-a870-486f-9d2f-3b33a6707b2b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.047802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bffe8e53f534c1ba596feccd9de7acb', 'request_id': 'req-39484f6d-a870-486f-9d2f-3b33a6707b2b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.054825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bffe8e53f534c1ba596feccd9de7acb', 'request_id': 'req-39484f6d-a870-486f-9d2f-3b33a6707b2b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.065946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] The token cache is being invalidate because project 1bffe8e53f534c1ba596feccd9de7acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:24.066259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.066870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.068444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 965/1944] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:23 2022] DELETE /identity/v3/projects/1bffe8e53f534c1ba596feccd9de7acb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:24.072392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.073832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.097356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.098029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.098313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.098587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.099316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:24.099642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.101282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.113251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 980/1945] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:14:24.118973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.120609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.133145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc0623-5745-4fa5-bf49-afaf52820031 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.136713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484f6d-a870-486f-9d2f-3b33a6707b2b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.138180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.138832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.139115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.139395 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.140104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.141841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.142088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.142727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.143009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.143282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] PATH_INFO: `/v3/users/5bbac03acca549158f3605f545659ffa` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.144529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bbac03acca549158f3605f545659ffa)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.146607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.160126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:24.173823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 981/1946] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:24 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:24.175424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.197726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bbac03acca549158f3605f545659ffa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.198407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bbac03acca549158f3605f545659ffa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.211065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bbac03acca549158f3605f545659ffa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.211734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bbac03acca549158f3605f545659ffa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.219081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bbac03acca549158f3605f545659ffa'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.233323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 966/1947] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:24 2022] DELETE /identity/v3/users/5bbac03acca549158f3605f545659ffa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:24.240560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195bb63-4b54-4f1d-8140-227e3969631d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.253793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:24.313596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.314473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.314841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.315186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.315993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.318496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.368733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 982/1948] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:24 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:24.370062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.371567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.375990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.379328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.388845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa778-8797-4e37-8a61-07cd8ebf9478 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.393842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.394526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.394805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.395279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.396015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.397782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9b4b3-2627-4646-9447-4cb577698b13 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.398150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.404544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.405585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.405870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.406353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] PATH_INFO: `/v3/users/ef2b86c3ebe04b65b81a8434e9709bb9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.408409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2b86c3ebe04b65b81a8434e9709bb9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.413340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 967/1949] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:24.419310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.419974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.441270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4617e2-d190-4a43-8e68-1cf8d09384b8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.450495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.451180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.451463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.451845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.452581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.454237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.467199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 968/1950] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:24.471630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.482748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee5ac-afcb-430a-a44a-8aa7403a721c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.482938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2b86c3ebe04b65b81a8434e9709bb9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.483577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2b86c3ebe04b65b81a8434e9709bb9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.484192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2b86c3ebe04b65b81a8434e9709bb9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.488083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9cc68-9583-4205-a494-d881f2d1111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.489437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9cc68-9583-4205-a494-d881f2d1111a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.490184 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9cc68-9583-4205-a494-d881f2d1111a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.490645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9cc68-9583-4205-a494-d881f2d1111a admin admin] PATH_INFO: `/v3/projects/a120b9dcdea447b5846bba82f2d407cd/users/54220270308d4119b617864641e0a0df/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.497103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2b86c3ebe04b65b81a8434e9709bb9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.504661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-83b6a91b-61a6-4abf-92c0-5a2e270d3b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2b86c3ebe04b65b81a8434e9709bb9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.511121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9cc68-9583-4205-a494-d881f2d1111a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a120b9dcdea447b5846bba82f2d407cd, user_id=54220270308d4119b617864641e0a0df, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.513855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9cc68-9583-4205-a494-d881f2d1111a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.522831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 983/1951] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:24 2022] DELETE /identity/v3/users/ef2b86c3ebe04b65b81a8434e9709bb9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:24.527616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.528175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.528625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.529356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 969/1952] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:24 2022] PUT /identity/v3/projects/a120b9dcdea447b5846bba82f2d407cd/users/54220270308d4119b617864641e0a0df/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:24.540991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.542187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.542187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.569359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:24.571424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.573710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] MFA Rules not processed for user `54220270308d4119b617864641e0a0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:24.575953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.655949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 970/1953] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:24.660150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 984/1954] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:24.666982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.667374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.668480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.668682 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.692104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:24.713433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:24.739509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4ebbd-861e-4477-a17c-098589ba2e51 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.741736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c333d-e6d1-44d0-b1f6-c50b71e1b51c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.744834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.745480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.745754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.746143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.746941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebf252-cfce-4756-808c-ea5637a63e07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.747496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf252-cfce-4756-808c-ea5637a63e07 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.747776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf252-cfce-4756-808c-ea5637a63e07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.748086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf252-cfce-4756-808c-ea5637a63e07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.756325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.758117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebf252-cfce-4756-808c-ea5637a63e07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.758421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.760135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebf252-cfce-4756-808c-ea5637a63e07 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.771101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 985/1955] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:24.772192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:24.773150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 971/1956] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:24.775618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:24.792150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2c8b7-c7d2-490c-b5e3-f26c8d1d50db service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:24.804588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:24.805738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.806087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.806429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] PATH_INFO: `/v3/projects/a73153785f1f4d8ea88a9e18a86a77b2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:24.811071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73153785f1f4d8ea88a9e18a86a77b2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:24.813433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:24.931961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73153785f1f4d8ea88a9e18a86a77b2', 'request_id': 'req-a623993f-6487-4cea-9970-6938641b7884'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.943877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73153785f1f4d8ea88a9e18a86a77b2', 'request_id': 'req-a623993f-6487-4cea-9970-6938641b7884'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.950815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73153785f1f4d8ea88a9e18a86a77b2', 'request_id': 'req-a623993f-6487-4cea-9970-6938641b7884'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.970727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] The token cache is being invalidate because project a73153785f1f4d8ea88a9e18a86a77b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:24.971018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.971688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a623993f-6487-4cea-9970-6938641b7884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:24.976709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 986/1957] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:24 2022] DELETE /identity/v3/projects/a73153785f1f4d8ea88a9e18a86a77b2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:24.978438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e80be-f13f-452b-b9dd-0d8e4bd2239a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:24.978862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e80be-f13f-452b-b9dd-0d8e4bd2239a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:24.979147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e80be-f13f-452b-b9dd-0d8e4bd2239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.018778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2b9e80be-f13f-452b-b9dd-0d8e4bd2239a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:25.020080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2b9e80be-f13f-452b-b9dd-0d8e4bd2239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:25.116598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 972/1958] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:25.124329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4882a-d755-4298-91cd-0de277e1e233 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.124887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4882a-d755-4298-91cd-0de277e1e233 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.125353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4882a-d755-4298-91cd-0de277e1e233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.146279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-28d4882a-d755-4298-91cd-0de277e1e233 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:25.147820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-28d4882a-d755-4298-91cd-0de277e1e233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:25.160711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 987/1959] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:25.161996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e80be-f13f-452b-b9dd-0d8e4bd2239a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.191207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e80be-f13f-452b-b9dd-0d8e4bd2239a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.191207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.191207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.191207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.191583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.191583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:25.191979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.193940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.212242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 973/1960] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:25.213159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4882a-d755-4298-91cd-0de277e1e233 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.226147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4882a-d755-4298-91cd-0de277e1e233 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.238278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.239087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.239267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.239524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.240365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.243328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.287573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 988/1961] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:25 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:25.289779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.301365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9c053-22a2-487c-83c0-38a25e5bfe1c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.363413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.364091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.364365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.364636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.365401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.367369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.388667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 974/1962] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:25 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:25.392891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.404433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4594c-7d12-4650-a553-c30509635dbd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.409516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.410295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.410652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.410944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.411659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.413805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.429103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 989/1963] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:25.429365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.431712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:25.432632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.444318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6a9e2-5c63-43db-a2ad-25e88d5df98a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.445626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f22b2-b701-4ba2-bac0-c50b5cc7bd43 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.450993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.452061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.452351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.452630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] PATH_INFO: `/v3/projects/a28d2292fb4d4e67a9baa599bc4c05f0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.453948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.454357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28d2292fb4d4e67a9baa599bc4c05f0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.454793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.455131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.455820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.456369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.456784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.458841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.473085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 990/1964] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:25.477398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.487611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7ee53-320b-4182-ba3e-ca84b27234d4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.492785 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bc7d7-c2c3-4259-aec7-5f19bb8079d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.493724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bc7d7-c2c3-4259-aec7-5f19bb8079d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.494007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bc7d7-c2c3-4259-aec7-5f19bb8079d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.494284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bc7d7-c2c3-4259-aec7-5f19bb8079d2 admin admin] PATH_INFO: `/v3/projects/fd74b86419ec498086d7d73275e2637d/users/c780f028ba0e40ae92302250e51877be/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.508919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:25.510474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28d2292fb4d4e67a9baa599bc4c05f0', 'request_id': 'req-82c09f5d-1ebb-4f26-9972-d5818393b2f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:25.510615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bc7d7-c2c3-4259-aec7-5f19bb8079d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd74b86419ec498086d7d73275e2637d, user_id=c780f028ba0e40ae92302250e51877be, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.513447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bc7d7-c2c3-4259-aec7-5f19bb8079d2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.521797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28d2292fb4d4e67a9baa599bc4c05f0', 'request_id': 'req-82c09f5d-1ebb-4f26-9972-d5818393b2f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:25.529283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 991/1965] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:25 2022] PUT /identity/v3/projects/fd74b86419ec498086d7d73275e2637d/users/c780f028ba0e40ae92302250e51877be/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:25.529661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28d2292fb4d4e67a9baa599bc4c05f0', 'request_id': 'req-82c09f5d-1ebb-4f26-9972-d5818393b2f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:25.540550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a67895-72a7-4db3-a247-58ffed69291f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.540839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a67895-72a7-4db3-a247-58ffed69291f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.541155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a67895-72a7-4db3-a247-58ffed69291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.541260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] The token cache is being invalidate because project a28d2292fb4d4e67a9baa599bc4c05f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:25.541539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:25.542208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:25.544154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 975/1966] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:25 2022] DELETE /identity/v3/projects/a28d2292fb4d4e67a9baa599bc4c05f0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:25.572624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-14a67895-72a7-4db3-a247-58ffed69291f None None] MFA Rules not processed for user `c780f028ba0e40ae92302250e51877be`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:25.573994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-14a67895-72a7-4db3-a247-58ffed69291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:25.598078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:25.677774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 992/1967] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:25.682741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.684427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:25.777180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c09f5d-1ebb-4f26-9972-d5818393b2f0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.836265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.836967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.837281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.837546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.838246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.840136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.868196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 976/1968] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:25 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:25.872510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a67895-72a7-4db3-a247-58ffed69291f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:25.884631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a67895-72a7-4db3-a247-58ffed69291f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:25.955306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:25.956117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:25.956401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:25.956675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:25.957427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:25.959399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:25.981040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 993/1969] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:25 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:25.986405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:26.006078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fc0ba-2eb9-477b-a9ba-42cdf88ffd31 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:26.011777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:26.012466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.012756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.013648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.013974 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:26.016294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:26.029144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 977/1970] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:26.033439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:26.044513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8183290-9584-45e6-9260-d1cb6657ddab admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:26.049491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:26.050560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.050931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.051299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] PATH_INFO: `/v3/projects/0d1c591008cc4b679c4b1fa0045affd6/users/8bf602494dd548a6849c666e0a3fd68f/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.069307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1c591008cc4b679c4b1fa0045affd6, user_id=8bf602494dd548a6849c666e0a3fd68f, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:26.072066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:26.086142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 994/1971] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:26 2022] PUT /identity/v3/projects/0d1c591008cc4b679c4b1fa0045affd6/users/8bf602494dd548a6849c666e0a3fd68f/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:26.090060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:26.100204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951a20-0fee-43d9-bcba-07bb28a46b28 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:26.155571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:26.163812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf188bc-5597-4447-bad6-5f0d0db46723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:26.164860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf188bc-5597-4447-bad6-5f0d0db46723 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.165370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf188bc-5597-4447-bad6-5f0d0db46723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.166382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf188bc-5597-4447-bad6-5f0d0db46723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.167614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf188bc-5597-4447-bad6-5f0d0db46723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:26.170458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf188bc-5597-4447-bad6-5f0d0db46723 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:26.184788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 978/1972] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:26.190302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:26.202107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896d443-3852-4719-b30c-d2536c0968f6 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:26.206954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:26.207856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.208138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.208415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] PATH_INFO: `/v3/projects/0d1c591008cc4b679c4b1fa0045affd6/users/8bf602494dd548a6849c666e0a3fd68f/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.210504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1c591008cc4b679c4b1fa0045affd6, user_id=8bf602494dd548a6849c666e0a3fd68f, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:26.213167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:26.223637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 995/1973] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:26 2022] PUT /identity/v3/projects/0d1c591008cc4b679c4b1fa0045affd6/users/8bf602494dd548a6849c666e0a3fd68f/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:26.234091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d7ed-66cd-4eaa-b57b-d3ae792656d3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.234742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d7ed-66cd-4eaa-b57b-d3ae792656d3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.234887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d7ed-66cd-4eaa-b57b-d3ae792656d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.264297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a346d7ed-66cd-4eaa-b57b-d3ae792656d3 None None] MFA Rules not processed for user `8bf602494dd548a6849c666e0a3fd68f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:26.265560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a346d7ed-66cd-4eaa-b57b-d3ae792656d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:26.337157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 979/1974] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:26.351865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:26.354056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:26.422734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac13ed-55c7-400a-a0ed-825cc0528d0d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:26.426507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13bf3b-4c95-4500-83a5-021df58bce68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:26.427166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13bf3b-4c95-4500-83a5-021df58bce68 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.427458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13bf3b-4c95-4500-83a5-021df58bce68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.427848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13bf3b-4c95-4500-83a5-021df58bce68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.440280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13bf3b-4c95-4500-83a5-021df58bce68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:26.441717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13bf3b-4c95-4500-83a5-021df58bce68 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:26.457175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 996/1975] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:26.536095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346d7ed-66cd-4eaa-b57b-d3ae792656d3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:26.605032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346d7ed-66cd-4eaa-b57b-d3ae792656d3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:26.610378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:26.611383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.611715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.611993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] PATH_INFO: `/v3/users/c780f028ba0e40ae92302250e51877be` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.613277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c780f028ba0e40ae92302250e51877be)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:26.615299 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:26.665769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c780f028ba0e40ae92302250e51877be'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:26.666537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c780f028ba0e40ae92302250e51877be'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:26.683199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c780f028ba0e40ae92302250e51877be'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:26.683905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c780f028ba0e40ae92302250e51877be'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:26.691355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c780f028ba0e40ae92302250e51877be'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:26.703794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:26.710568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 980/1976] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:26 2022] DELETE /identity/v3/users/c780f028ba0e40ae92302250e51877be => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:26.715426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.715720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.716167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.734145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:26.735354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:26.746166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:26.802144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 997/1977] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:26.813680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:26.815307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:26.874385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52fe96-af92-41ac-bcd6-d5ffb1c21b2f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:26.878359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:26.879014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:26.879714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:26.879714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:26.892049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:26.893388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:26.907969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 981/1978] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:27.194997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:27.197119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:27.212313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb4d78-b77e-4a9d-a411-f61de7636cca None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:27.218396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:27.219335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:27.219617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:27.219896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] PATH_INFO: `/v3/projects/fd74b86419ec498086d7d73275e2637d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:27.221554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd74b86419ec498086d7d73275e2637d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:27.223483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:27.276177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd74b86419ec498086d7d73275e2637d', 'request_id': 'req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.282762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd74b86419ec498086d7d73275e2637d', 'request_id': 'req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.288341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd74b86419ec498086d7d73275e2637d', 'request_id': 'req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.302636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] The token cache is being invalidate because project fd74b86419ec498086d7d73275e2637d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:27.303142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.303902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.305624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 998/1979] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:27 2022] DELETE /identity/v3/projects/fd74b86419ec498086d7d73275e2637d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:27.309718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:27.319529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca9aa-16b5-46ac-a7b2-1622a1ac8c0b service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:27.375943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:27.376817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:27.377122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:27.377705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] PATH_INFO: `/v3/users/8bf602494dd548a6849c666e0a3fd68f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:27.379025 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf602494dd548a6849c666e0a3fd68f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:27.380964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:27.432740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf602494dd548a6849c666e0a3fd68f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.433684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf602494dd548a6849c666e0a3fd68f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.447341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf602494dd548a6849c666e0a3fd68f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.448029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf602494dd548a6849c666e0a3fd68f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.455854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-49647a2e-8482-46fa-8410-acdad4bab5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf602494dd548a6849c666e0a3fd68f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:27.471103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 982/1980] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:27 2022] DELETE /identity/v3/users/8bf602494dd548a6849c666e0a3fd68f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:28.154759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.156372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:28.177436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80350b6-d8f7-4855-a97d-40f9bd0ba86f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.242660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.243698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.243972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.244250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] PATH_INFO: `/v3/projects/0d1c591008cc4b679c4b1fa0045affd6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.245495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1c591008cc4b679c4b1fa0045affd6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.247608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:28.290830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:28.297906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1c591008cc4b679c4b1fa0045affd6', 'request_id': 'req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:28.304901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1c591008cc4b679c4b1fa0045affd6', 'request_id': 'req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:28.311325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1c591008cc4b679c4b1fa0045affd6', 'request_id': 'req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:28.325426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] The token cache is being invalidate because project 0d1c591008cc4b679c4b1fa0045affd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:28.325687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:28.326378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-06a162bf-1a38-4a5a-9e61-32f2ea3a2ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:28.331348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 999/1981] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:28 2022] DELETE /identity/v3/projects/0d1c591008cc4b679c4b1fa0045affd6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:28.333189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413159-a344-446c-9771-c83ada70331c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.333698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413159-a344-446c-9771-c83ada70331c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.333796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413159-a344-446c-9771-c83ada70331c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.353239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c3413159-a344-446c-9771-c83ada70331c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:28.354470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c3413159-a344-446c-9771-c83ada70331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:28.369072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:28.424572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 983/1982] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:28.432501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d07f9-79f8-4606-ac77-535873b75301 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.432874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d07f9-79f8-4606-ac77-535873b75301 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.433310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d07f9-79f8-4606-ac77-535873b75301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.452613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7d6d07f9-79f8-4606-ac77-535873b75301 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:28.453977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7d6d07f9-79f8-4606-ac77-535873b75301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:28.463403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1000/1983] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:28.468052 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3413159-a344-446c-9771-c83ada70331c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.478555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3413159-a344-446c-9771-c83ada70331c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.483425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.484135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.484668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.484952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.485723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:28.486030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.487819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:28.503382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 984/1984] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:28.508095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d07f9-79f8-4606-ac77-535873b75301 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.517720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d07f9-79f8-4606-ac77-535873b75301 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.524063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.524737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.525016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.525341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.526098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.528205 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:28.563516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1001/1985] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:28 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:28.566255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.579195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405916b7-ea13-4d71-a6d1-183e9d5dc70f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.636603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.637326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.637611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.637885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.638580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.640541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:28.662085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 985/1986] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:28 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:28.666352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.678155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8d0b-58fe-4c06-a155-ed0ac73a498c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.683312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.683999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.684296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.684570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.685354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.686980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:28.701399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1002/1987] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:28.702716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.713234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8e12-6541-48a5-b7d0-d52dad1feb95 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.718373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144587e-530a-4ecb-b05c-52fc9881a091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.719048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144587e-530a-4ecb-b05c-52fc9881a091 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.719334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144587e-530a-4ecb-b05c-52fc9881a091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.719612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144587e-530a-4ecb-b05c-52fc9881a091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.720350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144587e-530a-4ecb-b05c-52fc9881a091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.721990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144587e-530a-4ecb-b05c-52fc9881a091 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:28.733513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 986/1988] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:28.737318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.748286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c2272-e543-4661-95f0-94214771a842 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.753244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.754145 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.754431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.754745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] PATH_INFO: `/v3/projects/c44ccc00fd1e4c6a87eaff5fabf37ba3/users/e8beb423bbef4e65a21c1528a98a725f/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.772449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c44ccc00fd1e4c6a87eaff5fabf37ba3, user_id=e8beb423bbef4e65a21c1528a98a725f, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.776068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:28.802640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1003/1989] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:28 2022] PUT /identity/v3/projects/c44ccc00fd1e4c6a87eaff5fabf37ba3/users/e8beb423bbef4e65a21c1528a98a725f/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:28.810037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715cbd-feb5-4435-965f-d3755c60d1a2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.810618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715cbd-feb5-4435-965f-d3755c60d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.810752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715cbd-feb5-4435-965f-d3755c60d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.839470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-44715cbd-feb5-4435-965f-d3755c60d1a2 None None] MFA Rules not processed for user `e8beb423bbef4e65a21c1528a98a725f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:28.840755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-44715cbd-feb5-4435-965f-d3755c60d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:28.910359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 987/1990] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:28.915838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:28.983642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7e9b-2408-48bb-9a72-6b7d5988868b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:28.989133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:28.989870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:28.990250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:28.991255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:28.991998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:28.994050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.031942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1004/1991] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:28 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:29.033448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44715cbd-feb5-4435-965f-d3755c60d1a2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:29.045474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44715cbd-feb5-4435-965f-d3755c60d1a2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:29.064465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:29.114507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:29.115763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.116267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.116730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.118063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:29.120379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.145241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 988/1992] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:29 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:29.146964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:29.157900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241a169-4c30-49ed-8bd2-96f76e4ff79a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:29.163624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:29.164400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.164686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.164962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.165733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:29.167350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.180269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1005/1993] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:29.184537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:29.195404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa438abc-c118-4cef-9877-f2576fb9f0bd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:29.200082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:29.200999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.201315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.201590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] PATH_INFO: `/v3/projects/c27cbb2e600644c6a43daeba706e153c/users/364111ae557849f9b98e3cdeea3eda01/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.217812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27cbb2e600644c6a43daeba706e153c, user_id=364111ae557849f9b98e3cdeea3eda01, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:29.220512 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.235314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 989/1994] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:29 2022] PUT /identity/v3/projects/c27cbb2e600644c6a43daeba706e153c/users/364111ae557849f9b98e3cdeea3eda01/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:29.239328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:29.250663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a7e24-5a71-40f6-aff8-ebd36495e31a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:29.302761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:29.325322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0c27b-5076-4fa2-8374-c2db62aec8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:29.326007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c27b-5076-4fa2-8374-c2db62aec8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.326290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c27b-5076-4fa2-8374-c2db62aec8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.326566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0c27b-5076-4fa2-8374-c2db62aec8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.327331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0c27b-5076-4fa2-8374-c2db62aec8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:29.328931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0c27b-5076-4fa2-8374-c2db62aec8a0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.342160 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1006/1995] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:29.346362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:29.357405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c2419-83f1-4844-95d0-ce29c2a8af8f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:29.362609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:29.363678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.363971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.364429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] PATH_INFO: `/v3/projects/c27cbb2e600644c6a43daeba706e153c/users/364111ae557849f9b98e3cdeea3eda01/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.367412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27cbb2e600644c6a43daeba706e153c, user_id=364111ae557849f9b98e3cdeea3eda01, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:29.370138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.383072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 990/1996] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:29 2022] PUT /identity/v3/projects/c27cbb2e600644c6a43daeba706e153c/users/364111ae557849f9b98e3cdeea3eda01/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:29.393530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2a60-a871-495e-b2ff-76b73a30964e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.393969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2a60-a871-495e-b2ff-76b73a30964e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.394476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a2a60-a871-495e-b2ff-76b73a30964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.426390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-692a2a60-a871-495e-b2ff-76b73a30964e None None] MFA Rules not processed for user `364111ae557849f9b98e3cdeea3eda01`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:29.427691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-692a2a60-a871-495e-b2ff-76b73a30964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:29.497128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1007/1997] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:29.508711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:29.510726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:29.576486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c71e63-f7f2-45c4-95e5-aa8504afafe1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:29.580374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:29.581008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.581316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.581592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.592673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:29.594446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.605768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 991/1998] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:29.689761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a2a60-a871-495e-b2ff-76b73a30964e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:29.700766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a2a60-a871-495e-b2ff-76b73a30964e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:29.704479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b75c14-e109-4164-8c17-b83477af00c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:29.705158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b75c14-e109-4164-8c17-b83477af00c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:29.705412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b75c14-e109-4164-8c17-b83477af00c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:29.705687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b75c14-e109-4164-8c17-b83477af00c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:29.765745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b75c14-e109-4164-8c17-b83477af00c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:29.767232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b75c14-e109-4164-8c17-b83477af00c5 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:29.776567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1008/1999] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:33.479159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:33.480924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:33.557439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0646e-3889-47dd-be41-f391e709548e service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:33.562080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:33.562996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:33.563274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:33.563548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] PATH_INFO: `/v3/users/54220270308d4119b617864641e0a0df` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:33.564750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54220270308d4119b617864641e0a0df)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:33.566697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:33.580489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:33.618033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54220270308d4119b617864641e0a0df'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:33.618604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54220270308d4119b617864641e0a0df'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:33.631119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54220270308d4119b617864641e0a0df'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:33.631838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54220270308d4119b617864641e0a0df'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:33.640844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54220270308d4119b617864641e0a0df'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:33.655562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 992/2000] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:33 2022] DELETE /identity/v3/users/54220270308d4119b617864641e0a0df => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:33.660155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:33.660620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:33.660714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:33.678977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:33.680589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:33.734091 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:33.744745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1009/2001] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:33.754813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:33.756327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:33.817786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b38373-d42f-4e2a-a146-4b212f05c254 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:33.821580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174566a-8e6d-436e-a32a-0c23028a3531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:33.822242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174566a-8e6d-436e-a32a-0c23028a3531 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:33.822534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174566a-8e6d-436e-a32a-0c23028a3531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:33.822811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174566a-8e6d-436e-a32a-0c23028a3531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:33.832037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174566a-8e6d-436e-a32a-0c23028a3531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:33.833556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174566a-8e6d-436e-a32a-0c23028a3531 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:33.843577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 993/2002] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:34.135267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.136653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:34.149641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a78c73-81da-4327-aeb7-486248592c4e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.154615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.156044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.156044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.156044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] PATH_INFO: `/v3/projects/a120b9dcdea447b5846bba82f2d407cd` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.157287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a120b9dcdea447b5846bba82f2d407cd)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.159222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.208878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a120b9dcdea447b5846bba82f2d407cd', 'request_id': 'req-41ef1cd2-c248-46c1-9bdb-65dd092b5755'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.215483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a120b9dcdea447b5846bba82f2d407cd', 'request_id': 'req-41ef1cd2-c248-46c1-9bdb-65dd092b5755'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.221144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a120b9dcdea447b5846bba82f2d407cd', 'request_id': 'req-41ef1cd2-c248-46c1-9bdb-65dd092b5755'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.235698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] The token cache is being invalidate because project a120b9dcdea447b5846bba82f2d407cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:34.236020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.236615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-41ef1cd2-c248-46c1-9bdb-65dd092b5755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.238234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1010/2003] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:34 2022] DELETE /identity/v3/projects/a120b9dcdea447b5846bba82f2d407cd => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:34.243731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876469ba-8848-4b1c-aacd-c24727f8a816 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.244023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876469ba-8848-4b1c-aacd-c24727f8a816 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.244310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876469ba-8848-4b1c-aacd-c24727f8a816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.263015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-876469ba-8848-4b1c-aacd-c24727f8a816 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:34.264223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-876469ba-8848-4b1c-aacd-c24727f8a816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:34.325403 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 994/2004] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:34.333301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14d6f-52f6-4b0b-adb8-d46614b882da None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.333588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14d6f-52f6-4b0b-adb8-d46614b882da None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.333869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14d6f-52f6-4b0b-adb8-d46614b882da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.353575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4da14d6f-52f6-4b0b-adb8-d46614b882da None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:34.354817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4da14d6f-52f6-4b0b-adb8-d46614b882da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:34.363153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1011/2005] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:34.367454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876469ba-8848-4b1c-aacd-c24727f8a816 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.377143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876469ba-8848-4b1c-aacd-c24727f8a816 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.381842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.382501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.382895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.383156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.383882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:34.384187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.385897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.397719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 995/2006] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:34.402109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da14d6f-52f6-4b0b-adb8-d46614b882da None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.411729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da14d6f-52f6-4b0b-adb8-d46614b882da None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.417261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.417641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.417925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.418197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.419269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.421188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.449803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1012/2007] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:34 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:34.454425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.464922 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10415fd3-cf9b-4452-bd5a-dc00f23dc6b3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.482615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:34.531593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.532283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.532569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.532840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.533594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.535529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.560870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 996/2008] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:34 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:34.565162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.578053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3963535-f078-4c41-81a4-e9af2e048d2d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.582803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.583519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.583804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.584283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.584820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.586539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.602647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1013/2009] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:34.604320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.614778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b53e76-5649-4e77-af3c-4e2f9e09b35d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.619613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974825ea-e509-4c78-9f4f-862f27415266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.620285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974825ea-e509-4c78-9f4f-862f27415266 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.620563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974825ea-e509-4c78-9f4f-862f27415266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.620830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974825ea-e509-4c78-9f4f-862f27415266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.621586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974825ea-e509-4c78-9f4f-862f27415266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.623177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974825ea-e509-4c78-9f4f-862f27415266 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.633895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 997/2010] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:34.638094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.648404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f08dc3-8315-48bf-b5fe-916db1a4e3a0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.653536 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.654434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.654715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.655327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] PATH_INFO: `/v3/projects/04a2e6cc3cbd4691a33722b5d058e82b/users/516ee78e2f9c43b195ad64b7a1d68624/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.671849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a2e6cc3cbd4691a33722b5d058e82b, user_id=516ee78e2f9c43b195ad64b7a1d68624, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.674553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.690736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1014/2011] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:34 2022] PUT /identity/v3/projects/04a2e6cc3cbd4691a33722b5d058e82b/users/516ee78e2f9c43b195ad64b7a1d68624/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:34.695749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.696065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.696406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.714550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:34.715779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:34.762957 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.764774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:34.786820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 998/2012] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:34.799683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:34.801245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:34.838435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208162c3-5aad-4c38-82cb-df9c0c944c05 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.843041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.843933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.844217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.844490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] PATH_INFO: `/v3/users/e8beb423bbef4e65a21c1528a98a725f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.845712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8beb423bbef4e65a21c1528a98a725f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.847648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.858487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f73b0-5847-4943-985f-0cc131ad6873 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:34.862032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac337dc2-d343-40b1-838c-9da918a707d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:34.862681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac337dc2-d343-40b1-838c-9da918a707d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.862961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac337dc2-d343-40b1-838c-9da918a707d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.863275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac337dc2-d343-40b1-838c-9da918a707d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.870805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:34.874534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac337dc2-d343-40b1-838c-9da918a707d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:34.876621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac337dc2-d343-40b1-838c-9da918a707d4 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:34.889873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 999/2013] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:14:34.909530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8beb423bbef4e65a21c1528a98a725f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.910362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8beb423bbef4e65a21c1528a98a725f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.911078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8beb423bbef4e65a21c1528a98a725f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.924171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8beb423bbef4e65a21c1528a98a725f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.932105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8beb423bbef4e65a21c1528a98a725f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:34.946779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1015/2014] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:34 2022] DELETE /identity/v3/users/e8beb423bbef4e65a21c1528a98a725f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:34.951823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:34.952151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:34.952442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:34.972927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:34.974591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:35.021462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:35.045569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1000/2015] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:35.055797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:35.060472 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:35.123880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240476b-7fd9-4ecc-9884-3873ef9fc380 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:35.128039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:35.128701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:35.129035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:35.130869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:35.139962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:35.141266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:35.151990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1016/2016] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:35.440895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:35.442810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:35.461334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1512-8dfe-46d2-91c7-f85fce5e44b5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:35.467254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:35.468160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:35.468438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:35.468685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] PATH_INFO: `/v3/projects/c44ccc00fd1e4c6a87eaff5fabf37ba3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:35.469976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c44ccc00fd1e4c6a87eaff5fabf37ba3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:35.472109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:35.524071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c44ccc00fd1e4c6a87eaff5fabf37ba3', 'request_id': 'req-595e53a7-95f2-46b1-9f53-9096fe627938'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.534260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c44ccc00fd1e4c6a87eaff5fabf37ba3', 'request_id': 'req-595e53a7-95f2-46b1-9f53-9096fe627938'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.541073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c44ccc00fd1e4c6a87eaff5fabf37ba3', 'request_id': 'req-595e53a7-95f2-46b1-9f53-9096fe627938'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.552814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] The token cache is being invalidate because project c44ccc00fd1e4c6a87eaff5fabf37ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:35.553126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.553874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.555703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1001/2017] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:35 2022] DELETE /identity/v3/projects/c44ccc00fd1e4c6a87eaff5fabf37ba3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:35.559828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:35.570098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bd27-8272-4c57-9942-49e7f0e5a281 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:35.638937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:35.639844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:35.640127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:35.640405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] PATH_INFO: `/v3/users/364111ae557849f9b98e3cdeea3eda01` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:35.641685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=364111ae557849f9b98e3cdeea3eda01)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:35.643717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:35.696832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:35.705427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364111ae557849f9b98e3cdeea3eda01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.706839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364111ae557849f9b98e3cdeea3eda01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.707949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364111ae557849f9b98e3cdeea3eda01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.722935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364111ae557849f9b98e3cdeea3eda01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.733166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a23d7be0-687a-478c-ac7b-9a04bd686860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364111ae557849f9b98e3cdeea3eda01'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:35.750688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1017/2018] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:35 2022] DELETE /identity/v3/users/364111ae557849f9b98e3cdeea3eda01 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:36.165297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:36.169032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:36.193157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e53a7-95f2-46b1-9f53-9096fe627938 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:36.268148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:36.268731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.269120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.269471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] PATH_INFO: `/v3/projects/c27cbb2e600644c6a43daeba706e153c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.270964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c27cbb2e600644c6a43daeba706e153c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:36.273108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:36.334623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c27cbb2e600644c6a43daeba706e153c', 'request_id': 'req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:36.344581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c27cbb2e600644c6a43daeba706e153c', 'request_id': 'req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:36.353691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c27cbb2e600644c6a43daeba706e153c', 'request_id': 'req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:36.360287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:36.371511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] The token cache is being invalidate because project c27cbb2e600644c6a43daeba706e153c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:36.372711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:36.374551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2fe1f398-d385-4ca1-be8e-8c7d9ab8bcc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:36.376742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1002/2019] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:36 2022] DELETE /identity/v3/projects/c27cbb2e600644c6a43daeba706e153c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:36.381845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4fae-445e-4011-a225-85386995b8ab None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.382167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4fae-445e-4011-a225-85386995b8ab None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.383012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4fae-445e-4011-a225-85386995b8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.407070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0a1a4fae-445e-4011-a225-85386995b8ab None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:36.408698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0a1a4fae-445e-4011-a225-85386995b8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:36.505555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1018/2020] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:36.513564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb01f2-9db8-4170-baa6-6c9464fa24bd None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.513855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb01f2-9db8-4170-baa6-6c9464fa24bd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.514105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb01f2-9db8-4170-baa6-6c9464fa24bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.535094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4ffb01f2-9db8-4170-baa6-6c9464fa24bd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:36.537002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4ffb01f2-9db8-4170-baa6-6c9464fa24bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:36.549926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1003/2021] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:36.553912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a4fae-445e-4011-a225-85386995b8ab None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:36.565670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a4fae-445e-4011-a225-85386995b8ab None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:36.573289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:36.573289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.573289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.573289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.575687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:36.575687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:36.575687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:36.603794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1019/2022] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:14:36.607642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb01f2-9db8-4170-baa6-6c9464fa24bd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:36.617392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb01f2-9db8-4170-baa6-6c9464fa24bd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:36.622333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:36.623017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.623302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.623576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.624277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:36.628960 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:36.662778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1004/2023] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:36 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:36.667983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:36.681224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17825862-1bb8-4a01-9ceb-cda00c2e8b45 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:36.769627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:36.770581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.770868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.771142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.771870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:36.773999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:36.825222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1020/2024] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:36 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:36.826674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:36.843359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6779-3269-4cd6-85a3-7b76b9d13cc3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:36.848485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:36.849283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.849565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.850006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.853083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:36.854950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:36.874604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:36.875722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1005/2025] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:36.891460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db333ce-207e-4bef-b4a4-c670d6bada74 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:36.899541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8080b-f006-4738-9573-88de49e49231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:36.900959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8080b-f006-4738-9573-88de49e49231 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.901561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8080b-f006-4738-9573-88de49e49231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.902138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8080b-f006-4738-9573-88de49e49231 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.903510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8080b-f006-4738-9573-88de49e49231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:36.906109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8080b-f006-4738-9573-88de49e49231 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:36.920520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1021/2026] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:36.924866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:36.938135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cde3d-431e-4bf9-b0fb-06bab99f33a3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:36.943694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:36.945223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:36.945223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:36.946169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] PATH_INFO: `/v3/projects/aae343bc44064b14b5bc781b7f546679/users/9442077526394077aa1152771ae1ddd9/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:36.978855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae343bc44064b14b5bc781b7f546679, user_id=9442077526394077aa1152771ae1ddd9, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:36.982742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:37.010029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1006/2027] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:36 2022] PUT /identity/v3/projects/aae343bc44064b14b5bc781b7f546679/users/9442077526394077aa1152771ae1ddd9/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:37.021334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd547-77b0-4613-a9f8-0f47832904e6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.021334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd547-77b0-4613-a9f8-0f47832904e6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.021681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd547-77b0-4613-a9f8-0f47832904e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.050932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7afdd547-77b0-4613-a9f8-0f47832904e6 None None] MFA Rules not processed for user `9442077526394077aa1152771ae1ddd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:37.052166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7afdd547-77b0-4613-a9f8-0f47832904e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:37.073390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:37.157820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1022/2028] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:37.159173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:37.255837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce99e2e-3692-4b90-9c7c-cda31d5d516d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:37.262730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:37.263428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.263709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.263985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.264697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:37.266698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:37.312204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1007/2029] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:37 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:37.316476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd547-77b0-4613-a9f8-0f47832904e6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:37.336659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd547-77b0-4613-a9f8-0f47832904e6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:37.413405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:37.414155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.414631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.415090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.415806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:37.417827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:37.451315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1023/2030] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:37 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:37.455692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:37.471592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39fcc8-14c9-4f82-a794-c43d72ea6e26 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:37.478574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:37.479679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.480172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.480736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.482528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:37.485529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:37.501284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1008/2031] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:37.505174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:37.519902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93747b5a-62a3-42f3-8296-e5875d16d43a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:37.530812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:37.531743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.532030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.532313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] PATH_INFO: `/v3/projects/2ace3d3dcc2147f89d0f5df7c8de975a/users/541e460f0ad74e88ac9f83fd489bc140/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.552068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ace3d3dcc2147f89d0f5df7c8de975a, user_id=541e460f0ad74e88ac9f83fd489bc140, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:37.555449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:37.577337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1024/2032] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:37 2022] PUT /identity/v3/projects/2ace3d3dcc2147f89d0f5df7c8de975a/users/541e460f0ad74e88ac9f83fd489bc140/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:37.578844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:37.602081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9614819-574c-4884-b89f-255c8daec83e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:37.750740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f69e97-ec4c-48f8-ab8e-ee127a6eebb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:37.751504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f69e97-ec4c-48f8-ab8e-ee127a6eebb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.752245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f69e97-ec4c-48f8-ab8e-ee127a6eebb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.752535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f69e97-ec4c-48f8-ab8e-ee127a6eebb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.753319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f69e97-ec4c-48f8-ab8e-ee127a6eebb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:37.755537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f69e97-ec4c-48f8-ab8e-ee127a6eebb5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:37.797935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1009/2033] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:37.801986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:37.820524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d388c-a6bb-4c7a-b257-198f340a7b1e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:37.834547 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:37.838425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.839478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.839779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] PATH_INFO: `/v3/projects/2ace3d3dcc2147f89d0f5df7c8de975a/users/541e460f0ad74e88ac9f83fd489bc140/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.846462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ace3d3dcc2147f89d0f5df7c8de975a, user_id=541e460f0ad74e88ac9f83fd489bc140, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:37.851813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:37.892546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1025/2034] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:37 2022] PUT /identity/v3/projects/2ace3d3dcc2147f89d0f5df7c8de975a/users/541e460f0ad74e88ac9f83fd489bc140/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:37.896679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3731d-84ba-410c-afc5-eb014e4b13d5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:37.897082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3731d-84ba-410c-afc5-eb014e4b13d5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:37.897813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3731d-84ba-410c-afc5-eb014e4b13d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:37.905729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:37.952964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1bd3731d-84ba-410c-afc5-eb014e4b13d5 None None] MFA Rules not processed for user `541e460f0ad74e88ac9f83fd489bc140`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:37.954405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1bd3731d-84ba-410c-afc5-eb014e4b13d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:38.111260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1010/2035] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:38.122601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:38.124591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:38.223923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeaff9-a17e-40c4-87c1-f8e8d85fe4b4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:38.229294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9b85c-bfd5-4079-8f6e-cc35448ff5af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:38.229988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b85c-bfd5-4079-8f6e-cc35448ff5af service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:38.230284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b85c-bfd5-4079-8f6e-cc35448ff5af service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:38.230550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b85c-bfd5-4079-8f6e-cc35448ff5af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:38.246063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9b85c-bfd5-4079-8f6e-cc35448ff5af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:38.248412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9b85c-bfd5-4079-8f6e-cc35448ff5af service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:38.266451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1026/2036] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:38.836065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3731d-84ba-410c-afc5-eb014e4b13d5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:38.932588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3731d-84ba-410c-afc5-eb014e4b13d5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:38.939070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:38.940189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:38.940522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:38.940802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] PATH_INFO: `/v3/users/9442077526394077aa1152771ae1ddd9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:38.942446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9442077526394077aa1152771ae1ddd9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:38.946098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:39.078705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9442077526394077aa1152771ae1ddd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:39.079819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9442077526394077aa1152771ae1ddd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:39.110809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9442077526394077aa1152771ae1ddd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:39.112626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9442077526394077aa1152771ae1ddd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:39.137260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9442077526394077aa1152771ae1ddd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:39.170417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1011/2037] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:38 2022] DELETE /identity/v3/users/9442077526394077aa1152771ae1ddd9 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:39.176185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:39.178188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:39.178766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:39.223427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:39.224648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:39.230990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:39.318772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1027/2038] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:39.321917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:39.323538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:39.340754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:39.406468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce657a93-b205-43b8-8319-b74dcab8492a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:39.415393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:39.416617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:39.416826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:39.417306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:39.430400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:39.431962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:39.446779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1012/2039] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:14:39.855824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:39.857592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:39.898491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae50f1b-071e-4b4b-acce-174edf5348e9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:39.905410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:39.906526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:39.906812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:39.907398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] PATH_INFO: `/v3/projects/aae343bc44064b14b5bc781b7f546679` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:39.908895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae343bc44064b14b5bc781b7f546679)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:39.911060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.021437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae343bc44064b14b5bc781b7f546679', 'request_id': 'req-e8822c33-39c8-4072-a65a-e342fc522f52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.029210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae343bc44064b14b5bc781b7f546679', 'request_id': 'req-e8822c33-39c8-4072-a65a-e342fc522f52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.062182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae343bc44064b14b5bc781b7f546679', 'request_id': 'req-e8822c33-39c8-4072-a65a-e342fc522f52'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.079870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] The token cache is being invalidate because project aae343bc44064b14b5bc781b7f546679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:40.080234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.080867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e8822c33-39c8-4072-a65a-e342fc522f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.082679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1028/2040] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:39 2022] DELETE /identity/v3/projects/aae343bc44064b14b5bc781b7f546679 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:40.087698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.102375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732b8cf-1430-42d4-b81e-f450ed0e3173 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.106908 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.107199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.107483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.141341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] MFA Rules not processed for user `516ee78e2f9c43b195ad64b7a1d68624`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:40.142585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:40.189810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.191002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.191285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.191561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] PATH_INFO: `/v3/users/541e460f0ad74e88ac9f83fd489bc140` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.192793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=541e460f0ad74e88ac9f83fd489bc140)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.194807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.237090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1029/2041] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:40.240546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.243191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:40.271899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a0e3b-ff66-499e-8207-0c047f042029 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.275789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541e460f0ad74e88ac9f83fd489bc140'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.276945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541e460f0ad74e88ac9f83fd489bc140'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.278856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.279958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.280235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.280509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.281257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.283722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.292917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541e460f0ad74e88ac9f83fd489bc140'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.294331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541e460f0ad74e88ac9f83fd489bc140'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.305537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541e460f0ad74e88ac9f83fd489bc140'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.325177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1030/2042] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:40 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:40.326140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.328498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:40.332839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1013/2043] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:40 2022] DELETE /identity/v3/users/541e460f0ad74e88ac9f83fd489bc140 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:40.344790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:40.457590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca36f-5246-4851-b711-5f237ac19c89 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.518917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.519713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.520048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.520328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.521031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.523029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.553872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1031/2044] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:40 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:40.555724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.566453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6615c9-8767-4dfd-8581-910d9ef628a9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.571363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.572046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.572324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.572594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.573533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.575155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.589891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1014/2045] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:40.591236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.602741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa5303-96f0-4242-a8e9-de76c4b2d0b7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.607948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.608938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.609538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.609752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] PATH_INFO: `/v3/projects/dbaa095c7963427f9a8cf59b1dd023e2/users/3404aa16454b4735a63758f87eb920a8/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.626772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbaa095c7963427f9a8cf59b1dd023e2, user_id=3404aa16454b4735a63758f87eb920a8, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.629554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.647295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1032/2046] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:40 2022] PUT /identity/v3/projects/dbaa095c7963427f9a8cf59b1dd023e2/users/3404aa16454b4735a63758f87eb920a8/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:40.651765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.663218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bb247-e145-4931-a34c-88503dc095b4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.678686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:40.696731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.699776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:40.736443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.737292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.737574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.737849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.738579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.740209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.750781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1015/2047] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:40.757026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:40.768489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d9ca6-a184-448f-be5d-0c9969511d8c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.773435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46734765-beaa-434b-9edc-0a8af657abf8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:40.773566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44458c6e-ffd7-4679-b839-86438eef82ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.774336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44458c6e-ffd7-4679-b839-86438eef82ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.774615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44458c6e-ffd7-4679-b839-86438eef82ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.774884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44458c6e-ffd7-4679-b839-86438eef82ec admin admin] PATH_INFO: `/v3/projects/dbaa095c7963427f9a8cf59b1dd023e2/users/3404aa16454b4735a63758f87eb920a8/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.776985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44458c6e-ffd7-4679-b839-86438eef82ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbaa095c7963427f9a8cf59b1dd023e2, user_id=3404aa16454b4735a63758f87eb920a8, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.779252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:40.779676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44458c6e-ffd7-4679-b839-86438eef82ec admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.780140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.780422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.781152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] PATH_INFO: `/v3/projects/2ace3d3dcc2147f89d0f5df7c8de975a` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.782523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ace3d3dcc2147f89d0f5df7c8de975a)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:40.784492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:40.790811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1016/2048] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:40 2022] PUT /identity/v3/projects/dbaa095c7963427f9a8cf59b1dd023e2/users/3404aa16454b4735a63758f87eb920a8/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:14:40.804564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:40.845454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ace3d3dcc2147f89d0f5df7c8de975a', 'request_id': 'req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.851749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ace3d3dcc2147f89d0f5df7c8de975a', 'request_id': 'req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.859034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ace3d3dcc2147f89d0f5df7c8de975a', 'request_id': 'req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.880945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] The token cache is being invalidate because project 2ace3d3dcc2147f89d0f5df7c8de975a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:40.883112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.884441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-dc5ca64d-dc07-4ac7-8e0c-d63489d891f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:40.892749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1033/2049] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:40 2022] DELETE /identity/v3/projects/2ace3d3dcc2147f89d0f5df7c8de975a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:40.898577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f6eeb-a8bc-41f4-baa8-07c6c536f021 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:40.898931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f6eeb-a8bc-41f4-baa8-07c6c536f021 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:40.899273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f6eeb-a8bc-41f4-baa8-07c6c536f021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:40.930027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c27f6eeb-a8bc-41f4-baa8-07c6c536f021 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:40.931646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c27f6eeb-a8bc-41f4-baa8-07c6c536f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:41.025107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1017/2050] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:41.032289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276db09-e5f2-475e-b886-23d482581fed None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.032635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276db09-e5f2-475e-b886-23d482581fed None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.032919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276db09-e5f2-475e-b886-23d482581fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.056667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f276db09-e5f2-475e-b886-23d482581fed None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:41.058002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f276db09-e5f2-475e-b886-23d482581fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:41.073267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1034/2051] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:41.079696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f6eeb-a8bc-41f4-baa8-07c6c536f021 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:41.093797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f6eeb-a8bc-41f4-baa8-07c6c536f021 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:41.102101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:41.103268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.103612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.103947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.104881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:41.105324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:41.107813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:41.129070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1018/2052] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:41.130779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276db09-e5f2-475e-b886-23d482581fed None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:41.155032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276db09-e5f2-475e-b886-23d482581fed None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:41.160773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:41.161493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.161916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.162195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.162902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:41.165467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:41.209093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1035/2053] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:41 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:41.213770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:41.225087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a3be7-0057-42ef-aee1-01b5e647ee43 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:41.292705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:41.294013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.294394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.294759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.295617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:41.297724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:41.328377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1019/2054] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:41 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:41.332372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:41.348197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde29234-e333-4551-9653-7b6e505056fc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:41.360152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:41.361036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.361356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.361635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.362371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:41.364206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:41.382595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1036/2055] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:41.389030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:41.404853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344dfea2-b5be-450b-8b79-27b45caf72c5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:41.409926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285721c-dc88-4eee-b4f7-9fb610fede32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:41.410591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285721c-dc88-4eee-b4f7-9fb610fede32 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.410873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285721c-dc88-4eee-b4f7-9fb610fede32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.411573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285721c-dc88-4eee-b4f7-9fb610fede32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.411908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285721c-dc88-4eee-b4f7-9fb610fede32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:41.413942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285721c-dc88-4eee-b4f7-9fb610fede32 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:41.428349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1020/2056] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:41.430059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:41.440565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccafcf4-a487-448a-b3fe-93bd4ce26490 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:41.445745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:41.446643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.447010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.447190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] PATH_INFO: `/v3/projects/055c2bf3d9dd4945b6eb8d9f343e4ee0/users/2233c079bf344d21abdb81b5fadb0563/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.463183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055c2bf3d9dd4945b6eb8d9f343e4ee0, user_id=2233c079bf344d21abdb81b5fadb0563, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:41.466513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:41.487605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1037/2057] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:41 2022] PUT /identity/v3/projects/055c2bf3d9dd4945b6eb8d9f343e4ee0/users/2233c079bf344d21abdb81b5fadb0563/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:41.498195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.498564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.498865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.515811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:41.533580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] MFA Rules not processed for user `2233c079bf344d21abdb81b5fadb0563`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:41.536095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:41.650963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1021/2058] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:41.665711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:41.667295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:41.741032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6f56f-ec8f-4e22-b894-8f4e4d27b3e3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:41.745293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:41.746114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:41.746395 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:41.746669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:41.757169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:41.759218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:41.771326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1038/2059] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:46.438930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:46.444241 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:46.546281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917ebea-a0d3-46cb-9e73-ab914ecf2c8d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:46.550824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:46.551799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:46.552079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:46.552354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] PATH_INFO: `/v3/users/8f1fe433179944b2b234ae9f164e8133` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:46.553611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1fe433179944b2b234ae9f164e8133)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:46.556098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:46.623861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1fe433179944b2b234ae9f164e8133'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:46.624620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1fe433179944b2b234ae9f164e8133'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:46.639041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1fe433179944b2b234ae9f164e8133'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:46.640049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1fe433179944b2b234ae9f164e8133'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:46.649425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d85ee36-2bba-4ad6-9102-0a4cedb9760a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1fe433179944b2b234ae9f164e8133'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:46.669066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1022/2060] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:46 2022] DELETE /identity/v3/users/8f1fe433179944b2b234ae9f164e8133 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:46.976852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:46.993184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308744ab-f8bd-444b-80ba-3402104fbd18 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.041620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.041913 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.042190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.062770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:47.078803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] MFA Rules not processed for user `3404aa16454b4735a63758f87eb920a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:47.081278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:47.084180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.085562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.085996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.086417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] PATH_INFO: `/v3/projects/c1156aa267ff456b849a9fd0c1a43357` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.087233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:47.089110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1156aa267ff456b849a9fd0c1a43357)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.091537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.146116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1156aa267ff456b849a9fd0c1a43357', 'request_id': 'req-72d7f18d-e96a-43ea-a03a-27200fc05dd8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:47.153810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1156aa267ff456b849a9fd0c1a43357', 'request_id': 'req-72d7f18d-e96a-43ea-a03a-27200fc05dd8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:47.161345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1156aa267ff456b849a9fd0c1a43357', 'request_id': 'req-72d7f18d-e96a-43ea-a03a-27200fc05dd8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:47.171536 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1023/2061] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:47.180677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] The token cache is being invalidate because project c1156aa267ff456b849a9fd0c1a43357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:47.181217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:47.182064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-72d7f18d-e96a-43ea-a03a-27200fc05dd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:47.183954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1039/2062] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:46 2022] DELETE /identity/v3/projects/c1156aa267ff456b849a9fd0c1a43357 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:47.187330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.187330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:47.190430 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295ed06-c6da-45e4-bbbf-6d239155e0ca None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.191224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295ed06-c6da-45e4-bbbf-6d239155e0ca None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.191612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295ed06-c6da-45e4-bbbf-6d239155e0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.221378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1295ed06-c6da-45e4-bbbf-6d239155e0ca None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:47.222784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1295ed06-c6da-45e4-bbbf-6d239155e0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:47.274850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27a7c-22aa-407c-ba33-913b22c5778d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.284913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1040/2063] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:47.293781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326866-bb7b-4b1d-b90b-c2bc8e3b1584 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.294077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326866-bb7b-4b1d-b90b-c2bc8e3b1584 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.294354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2326866-bb7b-4b1d-b90b-c2bc8e3b1584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.312580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c2326866-bb7b-4b1d-b90b-c2bc8e3b1584 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:47.313890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c2326866-bb7b-4b1d-b90b-c2bc8e3b1584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:47.322297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1041/2064] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:47.326356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2326866-bb7b-4b1d-b90b-c2bc8e3b1584 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.335589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2326866-bb7b-4b1d-b90b-c2bc8e3b1584 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.341919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.342590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.342872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.343149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.343883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:47.344191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.345856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.355718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.356401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.356818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.357128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.358485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1042/2065] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:14:47.358951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:47.362969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.374552 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b09108-6926-4c1d-808f-584f3998b923 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.379902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.380783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.381165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.381501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.382554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.384586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.399223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:47.415388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1043/2066] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:47 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:47.419601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.428398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.432366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.440587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1024/2067] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:47.458693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:47.505523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f13ca-71ae-4e46-bce2-4beda5299c33 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.517662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.518411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.518692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.518964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.519718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.522173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.558805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1044/2068] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:47 2022] POST /identity/v3/users => generated 303 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:14:47.562764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.575807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6a276-5ed5-4c30-9afd-92ded7102ea8 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.580103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.580793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.581168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.581449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.582182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.583801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.595855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1025/2069] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:47.600069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.614775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8d48f-7d90-4976-a9af-d44260a31297 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.622686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5a6c4-5e1f-42e6-bafc-0e9079472b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.622686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a6c4-5e1f-42e6-bafc-0e9079472b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.622686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a6c4-5e1f-42e6-bafc-0e9079472b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.623076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a6c4-5e1f-42e6-bafc-0e9079472b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.623653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5a6c4-5e1f-42e6-bafc-0e9079472b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.625313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5a6c4-5e1f-42e6-bafc-0e9079472b9d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.636419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1045/2070] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:47.640682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.653432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c26863-d603-4c41-9867-30f5313b9608 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.658632 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.659563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.659841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.660112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] PATH_INFO: `/v3/projects/c3c91283df3b4a6c937b103290632b1f/users/172515b2a1ec4e92a0eaf78e9656c425/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.679900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c91283df3b4a6c937b103290632b1f, user_id=172515b2a1ec4e92a0eaf78e9656c425, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:47.682763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:47.699625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1026/2071] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:47 2022] PUT /identity/v3/projects/c3c91283df3b4a6c937b103290632b1f/users/172515b2a1ec4e92a0eaf78e9656c425/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:47.709599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499fef-1537-4e23-ba9a-00b7374c20a4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.709886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499fef-1537-4e23-ba9a-00b7374c20a4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.710155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499fef-1537-4e23-ba9a-00b7374c20a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.743931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b8499fef-1537-4e23-ba9a-00b7374c20a4 None None] MFA Rules not processed for user `172515b2a1ec4e92a0eaf78e9656c425`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:47.749560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8499fef-1537-4e23-ba9a-00b7374c20a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:47.880535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1046/2072] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:47 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:47.884728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:47.983607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8893fd1-60cd-4180-bc8e-4ebf6b15f736 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:47.993464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:47.996659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:47.996805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:47.997218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:47.998049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:48.000257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:48.031174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1027/2073] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:47 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:48.035440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8499fef-1537-4e23-ba9a-00b7374c20a4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:48.049696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8499fef-1537-4e23-ba9a-00b7374c20a4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:48.126593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:48.127476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.127717 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.127993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.128757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:48.130941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:48.155785 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1047/2074] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:48 2022] POST /identity/v3/users => generated 308 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:48.160272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:48.171949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57f47-a11f-4502-8726-a8e27061e663 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:48.176930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:48.177685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.177969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.178242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.179086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:48.180904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:48.195649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1028/2075] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:48.200484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:48.211582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ba4d0-46ce-4e77-a6dc-01989baa1609 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:48.216825 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:48.217795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.218136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.218420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] PATH_INFO: `/v3/projects/9fa46cfc9f6349f4bdf0b20504c1905c/users/457fa34fb2c04218910b4e08d6c83c1c/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.239365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa46cfc9f6349f4bdf0b20504c1905c, user_id=457fa34fb2c04218910b4e08d6c83c1c, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:48.242858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:48.261696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1048/2076] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:48 2022] PUT /identity/v3/projects/9fa46cfc9f6349f4bdf0b20504c1905c/users/457fa34fb2c04218910b4e08d6c83c1c/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:48.265646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:48.276630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b90cc-0821-40ea-b5eb-4d3f23caee4d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:48.381883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edc3f5-2b3f-43e4-8027-71cbafade91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:48.382634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edc3f5-2b3f-43e4-8027-71cbafade91b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.382935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edc3f5-2b3f-43e4-8027-71cbafade91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.383261 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edc3f5-2b3f-43e4-8027-71cbafade91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.384061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edc3f5-2b3f-43e4-8027-71cbafade91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:48.385719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47edc3f5-2b3f-43e4-8027-71cbafade91b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:48.400222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1029/2077] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:48.404725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:48.416987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54c3ad-d635-475b-b821-96d1b15de80b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:48.422489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:48.423422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.423700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.423972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] PATH_INFO: `/v3/projects/9fa46cfc9f6349f4bdf0b20504c1905c/users/457fa34fb2c04218910b4e08d6c83c1c/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.426013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa46cfc9f6349f4bdf0b20504c1905c, user_id=457fa34fb2c04218910b4e08d6c83c1c, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:48.428657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:48.431502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:48.444641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1049/2078] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:48 2022] PUT /identity/v3/projects/9fa46cfc9f6349f4bdf0b20504c1905c/users/457fa34fb2c04218910b4e08d6c83c1c/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:48.455658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3839f73-b8ff-4552-8181-2c3fa5a7eb10 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.456190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3839f73-b8ff-4552-8181-2c3fa5a7eb10 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.456190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3839f73-b8ff-4552-8181-2c3fa5a7eb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.480296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:48.493630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c3839f73-b8ff-4552-8181-2c3fa5a7eb10 None None] MFA Rules not processed for user `457fa34fb2c04218910b4e08d6c83c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:48.495569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c3839f73-b8ff-4552-8181-2c3fa5a7eb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:48.582841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1030/2079] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:48.590943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:48.592728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:48.660077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578e41-1fa1-42a5-a68d-3cd1d1ef45f8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:48.664857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:48.665742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.666353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.666470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.677919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:48.679812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:48.690805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1050/2080] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:48 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:48.802997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80d9be-8e05-4546-a721-b792a5c3e8aa None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:48.803292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80d9be-8e05-4546-a721-b792a5c3e8aa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:48.803573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80d9be-8e05-4546-a721-b792a5c3e8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:48.825071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fd80d9be-8e05-4546-a721-b792a5c3e8aa None None] MFA Rules not processed for user `516ee78e2f9c43b195ad64b7a1d68624`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:48.826295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fd80d9be-8e05-4546-a721-b792a5c3e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:48.889591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1031/2081] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:51.058722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:51.130149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959f40b-7d0a-4400-bfc9-0ca5170c28f9 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:51.136153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:51.137168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:51.137386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:51.137661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] PATH_INFO: `/v3/users/172515b2a1ec4e92a0eaf78e9656c425` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:51.138847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=172515b2a1ec4e92a0eaf78e9656c425)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:51.140774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:51.193925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172515b2a1ec4e92a0eaf78e9656c425'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.195260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172515b2a1ec4e92a0eaf78e9656c425'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.196207 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172515b2a1ec4e92a0eaf78e9656c425'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.213823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172515b2a1ec4e92a0eaf78e9656c425'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.223792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172515b2a1ec4e92a0eaf78e9656c425'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.249250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1051/2082] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:51 2022] DELETE /identity/v3/users/172515b2a1ec4e92a0eaf78e9656c425 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:51.255428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:51.256082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:51.256356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:51.277428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:51.278766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:51.348947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1032/2083] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:51.359558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:51.361706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:51.387302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:51.451766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf4ee0-06e7-4a0f-86a9-8972f5def2ff admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:51.457406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:51.458678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:51.459296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:51.459905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:51.475031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:51.478193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:51.493012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1052/2084] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:51.836321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:51.838275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:51.863514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6dfe33-b484-423e-b4e0-99320a5effe8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:51.870392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:51.871363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:51.871650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:51.871926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] PATH_INFO: `/v3/projects/c3c91283df3b4a6c937b103290632b1f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:51.873283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c91283df3b4a6c937b103290632b1f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:51.877733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:51.924613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:51.935843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c91283df3b4a6c937b103290632b1f', 'request_id': 'req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.944926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c91283df3b4a6c937b103290632b1f', 'request_id': 'req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.951809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c91283df3b4a6c937b103290632b1f', 'request_id': 'req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.963799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] The token cache is being invalidate because project c3c91283df3b4a6c937b103290632b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:51.964087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.964765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:51.966615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1033/2085] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:51 2022] DELETE /identity/v3/projects/c3c91283df3b4a6c937b103290632b1f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:51.971111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:51.982466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237132a3-9ca7-4786-8671-d4b87ad5ca53 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:52.060880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:52.062054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:52.062343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:52.062627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] PATH_INFO: `/v3/users/457fa34fb2c04218910b4e08d6c83c1c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:52.064061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=457fa34fb2c04218910b4e08d6c83c1c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:52.066234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:52.123181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457fa34fb2c04218910b4e08d6c83c1c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.124166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457fa34fb2c04218910b4e08d6c83c1c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.124796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457fa34fb2c04218910b4e08d6c83c1c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.140009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457fa34fb2c04218910b4e08d6c83c1c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.147833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9edd90da-10e2-4d27-9fbf-88a9c53de488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457fa34fb2c04218910b4e08d6c83c1c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.180442 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1053/2086] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:14:51 2022] DELETE /identity/v3/users/457fa34fb2c04218910b4e08d6c83c1c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:52.641779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:52.643595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:52.675495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ee83c-fbe7-42fa-85b6-8736e94a5dd7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:52.790031 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:52.791266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:52.791550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:52.791819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] PATH_INFO: `/v3/projects/9fa46cfc9f6349f4bdf0b20504c1905c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:52.793467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa46cfc9f6349f4bdf0b20504c1905c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:52.795600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:52.857643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa46cfc9f6349f4bdf0b20504c1905c', 'request_id': 'req-47a51f3d-6449-4316-ae1f-a489c573d0a4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.868715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa46cfc9f6349f4bdf0b20504c1905c', 'request_id': 'req-47a51f3d-6449-4316-ae1f-a489c573d0a4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.880123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa46cfc9f6349f4bdf0b20504c1905c', 'request_id': 'req-47a51f3d-6449-4316-ae1f-a489c573d0a4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.892249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] The token cache is being invalidate because project 9fa46cfc9f6349f4bdf0b20504c1905c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:14:52.892787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.893576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-47a51f3d-6449-4316-ae1f-a489c573d0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:14:52.896853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1034/2087] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:14:52 2022] DELETE /identity/v3/projects/9fa46cfc9f6349f4bdf0b20504c1905c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:52.902473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d968d-b96c-4cdb-bbcf-ff5419ed5fa7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:52.902770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d968d-b96c-4cdb-bbcf-ff5419ed5fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:52.903048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d968d-b96c-4cdb-bbcf-ff5419ed5fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:52.926676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c73d968d-b96c-4cdb-bbcf-ff5419ed5fa7 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:52.928249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c73d968d-b96c-4cdb-bbcf-ff5419ed5fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:52.973450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:53.041249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1054/2088] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:53.049504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a1787-55c5-4082-86b0-4bce74ab20bf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.050065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a1787-55c5-4082-86b0-4bce74ab20bf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.050533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a1787-55c5-4082-86b0-4bce74ab20bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.072585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6d3a1787-55c5-4082-86b0-4bce74ab20bf None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:53.077097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6d3a1787-55c5-4082-86b0-4bce74ab20bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:53.093566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1035/2089] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:14:53.097972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d968d-b96c-4cdb-bbcf-ff5419ed5fa7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.111158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d968d-b96c-4cdb-bbcf-ff5419ed5fa7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.116116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.116893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.117240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.117520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.118264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:14:53.118574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.120271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.138420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a1787-55c5-4082-86b0-4bce74ab20bf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.142885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1055/2090] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:14:53.148898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a1787-55c5-4082-86b0-4bce74ab20bf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.154302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.155046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.155368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.155646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.156444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.159005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.212160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1036/2091] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:53 2022] POST /identity/v3/projects => generated 355 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:53.216373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.228353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727469f-9aa1-432b-873d-36b264d0f282 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.314647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.315411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.315736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.315936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.316642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.318636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.341752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1056/2092] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:53 2022] POST /identity/v3/users => generated 278 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:53.345959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.356770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeb6af-c649-48af-be12-23d4b08824e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.361480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.362224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.362565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.362901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.363706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.365381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.376850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1037/2093] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:53.381173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.391950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49eddd6-f1ec-42f0-a264-3cce39d0b446 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.396689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcf5e5-7920-46c2-a7c6-ec858f6e6175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.397447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcf5e5-7920-46c2-a7c6-ec858f6e6175 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.397710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcf5e5-7920-46c2-a7c6-ec858f6e6175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.397980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcf5e5-7920-46c2-a7c6-ec858f6e6175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.398735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcf5e5-7920-46c2-a7c6-ec858f6e6175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.400338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcf5e5-7920-46c2-a7c6-ec858f6e6175 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.411422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1057/2094] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:14:53.416243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.427160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c374c40-8d26-4c04-9485-294ed2d687d7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.432194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.433155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.433441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.433714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] PATH_INFO: `/v3/projects/c695bea5ab8e4a0cb0ef971b29566d36/users/b040a03ba66b4e36b8156abfe037bf42/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.446085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:53.449573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c695bea5ab8e4a0cb0ef971b29566d36, user_id=b040a03ba66b4e36b8156abfe037bf42, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.452693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.468895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1038/2095] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:53 2022] PUT /identity/v3/projects/c695bea5ab8e4a0cb0ef971b29566d36/users/b040a03ba66b4e36b8156abfe037bf42/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:53.479537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe6ded-51c8-46fc-ba66-80cea2da78c6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.479826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe6ded-51c8-46fc-ba66-80cea2da78c6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.480100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe6ded-51c8-46fc-ba66-80cea2da78c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.507920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2fbe6ded-51c8-46fc-ba66-80cea2da78c6 None None] MFA Rules not processed for user `b040a03ba66b4e36b8156abfe037bf42`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:53.509318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2fbe6ded-51c8-46fc-ba66-80cea2da78c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:53.578086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1058/2096] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:53 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:53.582832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.647758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cbeb-0c32-4504-b680-833bb29a9f26 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.652712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.653417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.653702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.653977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.654681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.656591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.687924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1039/2097] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:14:53 2022] POST /identity/v3/projects => generated 355 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:53.692464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe6ded-51c8-46fc-ba66-80cea2da78c6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.704476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe6ded-51c8-46fc-ba66-80cea2da78c6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.736930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:53.767901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.768636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.768979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.769383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.770147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.771981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.794373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1059/2098] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:14:53 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:14:53.798459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.810819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0ed73-1164-4060-9371-9c8595daf0b1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.818688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.819470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.820128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.820397 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.821274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.822912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.834366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1040/2099] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:53.839635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.850251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb6ecd-7c3b-4a4a-b9c9-208a08fca8d8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.855318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.856216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.856514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.856781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] PATH_INFO: `/v3/projects/400dfc39395f4cbe9e6e69a14badcf55/users/37f51a8d3689447ea0f3964e30d361ee/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.872013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=400dfc39395f4cbe9e6e69a14badcf55, user_id=37f51a8d3689447ea0f3964e30d361ee, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.874887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.890634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1060/2100] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:53 2022] PUT /identity/v3/projects/400dfc39395f4cbe9e6e69a14badcf55/users/37f51a8d3689447ea0f3964e30d361ee/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:53.894797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.905959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4926fa-7793-4541-b8de-163044a5c1c0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:53.966380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146acaad-6fb7-45ed-a51c-b37440d560f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:53.967078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146acaad-6fb7-45ed-a51c-b37440d560f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:53.967358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146acaad-6fb7-45ed-a51c-b37440d560f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:53.967627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146acaad-6fb7-45ed-a51c-b37440d560f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:53.968355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146acaad-6fb7-45ed-a51c-b37440d560f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:53.969989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146acaad-6fb7-45ed-a51c-b37440d560f7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:53.979717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1041/2101] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:14:53.984073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:53.995603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c392a12-e1db-45e4-9611-7184258dd2ac admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:54.002000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:54.002937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:54.003217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:54.003483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] PATH_INFO: `/v3/projects/400dfc39395f4cbe9e6e69a14badcf55/users/37f51a8d3689447ea0f3964e30d361ee/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:54.005709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=400dfc39395f4cbe9e6e69a14badcf55, user_id=37f51a8d3689447ea0f3964e30d361ee, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:54.008592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:54.020571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1061/2102] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:14:53 2022] PUT /identity/v3/projects/400dfc39395f4cbe9e6e69a14badcf55/users/37f51a8d3689447ea0f3964e30d361ee/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:14:54.034131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3907b-4ee3-48b8-a700-ffa61a826618 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:54.034454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3907b-4ee3-48b8-a700-ffa61a826618 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:54.034857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3907b-4ee3-48b8-a700-ffa61a826618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:54.066119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a2f3907b-4ee3-48b8-a700-ffa61a826618 None None] MFA Rules not processed for user `37f51a8d3689447ea0f3964e30d361ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:14:54.067248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a2f3907b-4ee3-48b8-a700-ffa61a826618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:54.119051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:14:54.144849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1042/2103] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:14:54.156691 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:54.159403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:14:54.226572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26869379-4b68-4718-88ba-021849f7db26 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:54.230661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:54.231412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:54.231696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:54.231972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:54.241964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:54.243417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:54.255009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1062/2104] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:54 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:14:54.336362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3907b-4ee3-48b8-a700-ffa61a826618 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:14:54.346699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3907b-4ee3-48b8-a700-ffa61a826618 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:14:54.350838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eac2dd-e0c4-4904-805d-b977103149b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:14:54.351494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eac2dd-e0c4-4904-805d-b977103149b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:14:54.351777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eac2dd-e0c4-4904-805d-b977103149b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:14:54.352308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eac2dd-e0c4-4904-805d-b977103149b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:14:54.419062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eac2dd-e0c4-4904-805d-b977103149b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:14:54.420551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eac2dd-e0c4-4904-805d-b977103149b0 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:14:54.431136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1043/2105] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:14:54 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:15:02.725147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:02.726718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:02.796340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88776fa-9a1b-4d3e-883b-0d7b6c53185d service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:02.801997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:02.802921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:02.803202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:02.803467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] PATH_INFO: `/v3/users/2233c079bf344d21abdb81b5fadb0563` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:02.804722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2233c079bf344d21abdb81b5fadb0563)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:02.806711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:02.848222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:02.863954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2233c079bf344d21abdb81b5fadb0563'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:02.864529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2233c079bf344d21abdb81b5fadb0563'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:02.865228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2233c079bf344d21abdb81b5fadb0563'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:02.877095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2233c079bf344d21abdb81b5fadb0563'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:02.884106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2233c079bf344d21abdb81b5fadb0563'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:02.898042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1063/2106] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:02 2022] DELETE /identity/v3/users/2233c079bf344d21abdb81b5fadb0563 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:02.902737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:02.903313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:02.903313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:02.922692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:02.924064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:02.987505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1044/2107] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:15:02.997337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:02.998817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:03.063389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf872a-d2e2-4017-9949-e8bd6ff2a8e2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:03.067660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7400070-2f84-4095-bb85-2442f5521f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:03.068373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400070-2f84-4095-bb85-2442f5521f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.068659 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400070-2f84-4095-bb85-2442f5521f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.068951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400070-2f84-4095-bb85-2442f5521f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.078829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7400070-2f84-4095-bb85-2442f5521f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:03.080274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7400070-2f84-4095-bb85-2442f5521f26 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:03.089874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1064/2108] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:03.448711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:03.450208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:03.462370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0ef52-18b9-4705-b68a-3b5b4b54804b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:03.470126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:03.471153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.471476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.471749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] PATH_INFO: `/v3/projects/055c2bf3d9dd4945b6eb8d9f343e4ee0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.473289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055c2bf3d9dd4945b6eb8d9f343e4ee0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:03.476456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:03.534991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055c2bf3d9dd4945b6eb8d9f343e4ee0', 'request_id': 'req-0ce06f57-4c15-470b-9379-abd2e5833f88'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:03.548852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055c2bf3d9dd4945b6eb8d9f343e4ee0', 'request_id': 'req-0ce06f57-4c15-470b-9379-abd2e5833f88'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:03.557886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055c2bf3d9dd4945b6eb8d9f343e4ee0', 'request_id': 'req-0ce06f57-4c15-470b-9379-abd2e5833f88'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:03.569469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] The token cache is being invalidate because project 055c2bf3d9dd4945b6eb8d9f343e4ee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:03.569763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:03.570422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0ce06f57-4c15-470b-9379-abd2e5833f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:03.571995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1045/2109] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:03 2022] DELETE /identity/v3/projects/055c2bf3d9dd4945b6eb8d9f343e4ee0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:03.577128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dca8fa-597e-41ad-92c4-c9b3781c9e20 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.578042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dca8fa-597e-41ad-92c4-c9b3781c9e20 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.578491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dca8fa-597e-41ad-92c4-c9b3781c9e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.598797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b8dca8fa-597e-41ad-92c4-c9b3781c9e20 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:03.599998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b8dca8fa-597e-41ad-92c4-c9b3781c9e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:03.670553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1065/2110] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:03.678645 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f3ff0-0932-4a6e-ada3-f6cd972593ed None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.678928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f3ff0-0932-4a6e-ada3-f6cd972593ed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.679209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f3ff0-0932-4a6e-ada3-f6cd972593ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.698283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-930f3ff0-0932-4a6e-ada3-f6cd972593ed None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:03.699440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-930f3ff0-0932-4a6e-ada3-f6cd972593ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:03.707700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1046/2111] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:03.713300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dca8fa-597e-41ad-92c4-c9b3781c9e20 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:03.723839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dca8fa-597e-41ad-92c4-c9b3781c9e20 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:03.733374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:03.734144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.734328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.734598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.735381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:15:03.735684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:03.737479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:03.750829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1066/2112] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:15:03.755783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f3ff0-0932-4a6e-ada3-f6cd972593ed None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:03.762211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:03.770426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f3ff0-0932-4a6e-ada3-f6cd972593ed None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:03.776905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:03.777503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.777756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.778171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.779125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:03.780696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:03.815185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1047/2113] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:15:03 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:03.823880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:03.836608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c263f-9a4e-4e18-9792-0640bfebf213 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:03.879316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:03.911648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:03.912458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.912799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.913103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.913809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:03.915780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:03.938488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1067/2114] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:15:03 2022] POST /identity/v3/users => generated 282 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:03.940460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:03.950690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301fed-4d1f-4362-a646-232dbe2360c9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:03.956044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:03.956712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.956990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.957300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.958045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:03.959655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:03.970540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1048/2115] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:03.974592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:03.984838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41948df-b6c6-4ff7-bdc3-c3154eec0483 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:03.990650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f5cae-917e-4f93-9be0-7c57a9442a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:03.991381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f5cae-917e-4f93-9be0-7c57a9442a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:03.991733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f5cae-917e-4f93-9be0-7c57a9442a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:03.992010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f5cae-917e-4f93-9be0-7c57a9442a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:03.993076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f5cae-917e-4f93-9be0-7c57a9442a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:03.995662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f5cae-917e-4f93-9be0-7c57a9442a92 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:04.006592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1068/2116] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:04.010928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:04.021852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2642be-a4c3-447a-9c64-cc537606416f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:04.027280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:04.028174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:04.028458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:04.028734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] PATH_INFO: `/v3/projects/2b71b7fd73dc4c8892afbc68ae2b99e9/users/2cf79a5122aa4f5e846a8252e03e60c5/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:04.044438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b71b7fd73dc4c8892afbc68ae2b99e9, user_id=2cf79a5122aa4f5e846a8252e03e60c5, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:04.047125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:04.060844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1049/2117] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:04 2022] PUT /identity/v3/projects/2b71b7fd73dc4c8892afbc68ae2b99e9/users/2cf79a5122aa4f5e846a8252e03e60c5/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:04.071980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:04.072356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:04.072715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:04.100979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] MFA Rules not processed for user `2cf79a5122aa4f5e846a8252e03e60c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:04.102282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:04.168409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1069/2118] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:04.182425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:04.184225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:04.244720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4104f28-efd2-43b6-9210-d36e6d0cd4e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:04.248710 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e388418-131b-42f2-be66-fe8d63f63e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:04.249434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e388418-131b-42f2-be66-fe8d63f63e54 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:04.249720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e388418-131b-42f2-be66-fe8d63f63e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:04.249993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e388418-131b-42f2-be66-fe8d63f63e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:04.259248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e388418-131b-42f2-be66-fe8d63f63e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:04.260677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e388418-131b-42f2-be66-fe8d63f63e54 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:04.270045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1050/2119] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:04 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:11.705005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:11.706556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:11.774246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6daa94-03ca-4569-90a6-9bc142cbf374 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:11.779701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:11.780595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:11.780886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:11.781201 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] PATH_INFO: `/v3/users/b040a03ba66b4e36b8156abfe037bf42` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:11.782548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=b040a03ba66b4e36b8156abfe037bf42)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:11.784550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:11.840229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b040a03ba66b4e36b8156abfe037bf42'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:11.840926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b040a03ba66b4e36b8156abfe037bf42'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:11.841592 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b040a03ba66b4e36b8156abfe037bf42'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:11.842755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:11.856386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b040a03ba66b4e36b8156abfe037bf42'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:11.863707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b040a03ba66b4e36b8156abfe037bf42'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:11.878551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1070/2120] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:11 2022] DELETE /identity/v3/users/b040a03ba66b4e36b8156abfe037bf42 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:11.883860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:11.884368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:11.884368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:11.902644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:11.903872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:11.965908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1051/2121] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:15:11.977456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:11.979141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:12.045077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711ab1c-f67d-4ede-bb03-1e32a32e69ed admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:12.049080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:12.049780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:12.050057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:12.050329 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:12.060515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:12.061988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:12.072352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1071/2122] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:12.367054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:12.369015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:12.381804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf0fc5-a1e6-4dfd-ba70-28afa861997b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:12.387162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:12.388794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:12.388794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:12.388794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] PATH_INFO: `/v3/projects/c695bea5ab8e4a0cb0ef971b29566d36` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:12.390039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c695bea5ab8e4a0cb0ef971b29566d36)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:12.391969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:12.440490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:12.445121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c695bea5ab8e4a0cb0ef971b29566d36', 'request_id': 'req-bc51dfcd-78e4-4438-bc0a-35d196d135c3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.453805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c695bea5ab8e4a0cb0ef971b29566d36', 'request_id': 'req-bc51dfcd-78e4-4438-bc0a-35d196d135c3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.464200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c695bea5ab8e4a0cb0ef971b29566d36', 'request_id': 'req-bc51dfcd-78e4-4438-bc0a-35d196d135c3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.478483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] The token cache is being invalidate because project c695bea5ab8e4a0cb0ef971b29566d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:12.478766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.479566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.481333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1052/2123] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:12 2022] DELETE /identity/v3/projects/c695bea5ab8e4a0cb0ef971b29566d36 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:12.486071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:12.498822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfa5ae-c69a-4564-b3ad-355c3927be83 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:12.558353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:12.559224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:12.559526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:12.559802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] PATH_INFO: `/v3/users/37f51a8d3689447ea0f3964e30d361ee` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:12.561030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f51a8d3689447ea0f3964e30d361ee)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:12.562993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:12.610309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f51a8d3689447ea0f3964e30d361ee'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.611300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f51a8d3689447ea0f3964e30d361ee'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.612003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f51a8d3689447ea0f3964e30d361ee'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.624793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f51a8d3689447ea0f3964e30d361ee'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.629300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:12.633668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-02ad7e1e-3cf7-4f7e-a7ce-6b785580b0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f51a8d3689447ea0f3964e30d361ee'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:12.648538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1072/2124] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:12 2022] DELETE /identity/v3/users/37f51a8d3689447ea0f3964e30d361ee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:12.979250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:12.980863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:12.999980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51dfcd-78e4-4438-bc0a-35d196d135c3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.058400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.059283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.059596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.059873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] PATH_INFO: `/v3/projects/400dfc39395f4cbe9e6e69a14badcf55` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.061019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=400dfc39395f4cbe9e6e69a14badcf55)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.062976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.111196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '400dfc39395f4cbe9e6e69a14badcf55', 'request_id': 'req-e5230fff-ca22-4577-9adc-d96f5ab4f893'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:13.120104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '400dfc39395f4cbe9e6e69a14badcf55', 'request_id': 'req-e5230fff-ca22-4577-9adc-d96f5ab4f893'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:13.126596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '400dfc39395f4cbe9e6e69a14badcf55', 'request_id': 'req-e5230fff-ca22-4577-9adc-d96f5ab4f893'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:13.137599 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] The token cache is being invalidate because project 400dfc39395f4cbe9e6e69a14badcf55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:13.137881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:13.138513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5230fff-ca22-4577-9adc-d96f5ab4f893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:13.140091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1053/2125] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:12 2022] DELETE /identity/v3/projects/400dfc39395f4cbe9e6e69a14badcf55 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:13.145314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c83c-795e-4a34-af69-633648a4ad20 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.145779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c83c-795e-4a34-af69-633648a4ad20 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.145871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c83c-795e-4a34-af69-633648a4ad20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.163938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6407c83c-795e-4a34-af69-633648a4ad20 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:13.165182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6407c83c-795e-4a34-af69-633648a4ad20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:13.225388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1073/2126] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:13.234069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4fafc-3be9-4680-9096-cee79dcffb6e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.234466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4fafc-3be9-4680-9096-cee79dcffb6e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.234654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4fafc-3be9-4680-9096-cee79dcffb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.258472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d0c4fafc-3be9-4680-9096-cee79dcffb6e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:13.258472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d0c4fafc-3be9-4680-9096-cee79dcffb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:13.265324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1054/2127] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:13.274017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407c83c-795e-4a34-af69-633648a4ad20 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.284258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407c83c-795e-4a34-af69-633648a4ad20 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.289959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.290641 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.290921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.291190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.292283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:15:13.292586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.294250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.306101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1074/2128] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:15:13.312520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4fafc-3be9-4680-9096-cee79dcffb6e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.326416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4fafc-3be9-4680-9096-cee79dcffb6e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.335095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.335753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.336035 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.336304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.336994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.338928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.368310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1055/2129] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:15:13 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:13.372570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.382799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4b9e0-c40c-4158-a21f-00840a1c0a0c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.445908 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.446861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.447203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.447576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.448588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.451605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.477073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1075/2130] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:15:13 2022] POST /identity/v3/users => generated 301 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:13.482681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.493747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644d513-8ca0-48f9-8fdb-d36eba741dbe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.499084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.499723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.499992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.500268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.501008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.502646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.514611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1056/2131] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:13.518761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.529874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa451aaa-091d-4b72-bdd1-49635c2e6b1e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.534843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cac61-f33d-4ee7-974c-1b024cead9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.535506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cac61-f33d-4ee7-974c-1b024cead9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.535794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cac61-f33d-4ee7-974c-1b024cead9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.536064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cac61-f33d-4ee7-974c-1b024cead9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.536798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cac61-f33d-4ee7-974c-1b024cead9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.538514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cac61-f33d-4ee7-974c-1b024cead9fb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.549576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1076/2132] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:13.554138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.564306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fc707-eb6a-4bbe-89fe-6080103e6c9b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.568891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.569837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.570112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.570382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] PATH_INFO: `/v3/projects/5946c10adfb44f1398e02c2165f83bb0/users/b2a8323f00834f76bd5cc228c75d9795/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.585148 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5946c10adfb44f1398e02c2165f83bb0, user_id=b2a8323f00834f76bd5cc228c75d9795, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.587762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.599294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:13.605735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1057/2133] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:13 2022] PUT /identity/v3/projects/5946c10adfb44f1398e02c2165f83bb0/users/b2a8323f00834f76bd5cc228c75d9795/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:13.616661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d85285-feb4-4c35-85c7-afe8a4203627 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.616948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d85285-feb4-4c35-85c7-afe8a4203627 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.617249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d85285-feb4-4c35-85c7-afe8a4203627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.645510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d1d85285-feb4-4c35-85c7-afe8a4203627 None None] MFA Rules not processed for user `b2a8323f00834f76bd5cc228c75d9795`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:13.646752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d1d85285-feb4-4c35-85c7-afe8a4203627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:13.689222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:13.714052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1077/2134] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:13.718713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.782142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9154e6d-6c02-48e8-85ef-fb2d92cf088c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.786991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.787652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.788039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.788360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.789174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.791068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.824667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1058/2135] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:15:13 2022] POST /identity/v3/projects => generated 399 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:15:13.825877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d85285-feb4-4c35-85c7-afe8a4203627 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.837240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d85285-feb4-4c35-85c7-afe8a4203627 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.897084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.897980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.898380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.898651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.899360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.901287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.922426 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1078/2136] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:15:13 2022] POST /identity/v3/users => generated 306 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:13.926402 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.936849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c54e7-80b7-4cf5-9202-33e5d17ac134 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.941997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.942713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.943038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.943381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.944107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:13.945840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:13.956844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1059/2137] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:13.960971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:13.972123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcb7cc-3653-4c85-bd4e-2513c6549f1f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:13.978071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:13.979660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:13.980104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:13.980656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] PATH_INFO: `/v3/projects/4bbf5df5f3934a9eb1186bfd19a6be11/users/360cf3ba97494f0e9764be2879121e09/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:13.999235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbf5df5f3934a9eb1186bfd19a6be11, user_id=360cf3ba97494f0e9764be2879121e09, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:14.002026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:14.018768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1079/2138] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:13 2022] PUT /identity/v3/projects/4bbf5df5f3934a9eb1186bfd19a6be11/users/360cf3ba97494f0e9764be2879121e09/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:15:14.024269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:14.035045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3b8e-7430-463b-ba32-41b170ac7946 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:14.102302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cf943-0c55-42e5-845b-a26db0d82358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:14.102971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf943-0c55-42e5-845b-a26db0d82358 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:14.103255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf943-0c55-42e5-845b-a26db0d82358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:14.103529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf943-0c55-42e5-845b-a26db0d82358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:14.104245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cf943-0c55-42e5-845b-a26db0d82358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:14.105895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cf943-0c55-42e5-845b-a26db0d82358 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:14.116062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1060/2139] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:15:14.120614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:14.131283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71e51a-3ec8-411b-9d33-d8d90b4cf13e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:14.136600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:14.137531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:14.137809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:14.138080 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] PATH_INFO: `/v3/projects/4bbf5df5f3934a9eb1186bfd19a6be11/users/360cf3ba97494f0e9764be2879121e09/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:14.140116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbf5df5f3934a9eb1186bfd19a6be11, user_id=360cf3ba97494f0e9764be2879121e09, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:14.142775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:14.153915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1080/2140] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:14 2022] PUT /identity/v3/projects/4bbf5df5f3934a9eb1186bfd19a6be11/users/360cf3ba97494f0e9764be2879121e09/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:14.165115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d1b6-cebe-4fe4-b31b-28a72e3879b5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:14.165424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d1b6-cebe-4fe4-b31b-28a72e3879b5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:14.165726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d1b6-cebe-4fe4-b31b-28a72e3879b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:14.192739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-49f0d1b6-cebe-4fe4-b31b-28a72e3879b5 None None] MFA Rules not processed for user `360cf3ba97494f0e9764be2879121e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:14.193981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-49f0d1b6-cebe-4fe4-b31b-28a72e3879b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:14.255174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:14.269769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1061/2141] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:14.278059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:14.279827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:14.339445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0f762-4dce-4806-aead-e5e1e6fd3f7b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:14.343534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:14.344177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:14.344451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:14.344734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:14.355071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:14.356551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:14.366182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1081/2142] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:14 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:14.450461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0d1b6-cebe-4fe4-b31b-28a72e3879b5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:14.460738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0d1b6-cebe-4fe4-b31b-28a72e3879b5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:14.464982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1d072-c852-46c8-938d-7d904a5735df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:14.465656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1d072-c852-46c8-938d-7d904a5735df service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:14.465940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1d072-c852-46c8-938d-7d904a5735df service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:14.466215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1d072-c852-46c8-938d-7d904a5735df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:14.524608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1d072-c852-46c8-938d-7d904a5735df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:14.525978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1d072-c852-46c8-938d-7d904a5735df service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:14.535005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1062/2143] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:18.665414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:18.709910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:18.733823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd88d69-753f-48e7-96ba-969425f0cf11 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:18.738724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:18.739614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:18.739893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:18.740163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] PATH_INFO: `/v3/users/b2a8323f00834f76bd5cc228c75d9795` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:18.741388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a8323f00834f76bd5cc228c75d9795)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:18.743364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:18.792613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a8323f00834f76bd5cc228c75d9795'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:18.793272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a8323f00834f76bd5cc228c75d9795'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:18.793865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a8323f00834f76bd5cc228c75d9795'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:18.806007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a8323f00834f76bd5cc228c75d9795'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:18.813110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a8323f00834f76bd5cc228c75d9795'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:18.826999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1082/2144] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:18 2022] DELETE /identity/v3/users/b2a8323f00834f76bd5cc228c75d9795 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:18.831769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:18.832257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:18.832352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:18.851448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:18.852671 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:18.916065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1063/2145] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:18.929060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:18.930671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:18.989079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0817bc8-bc49-4401-a855-f8707c00f5db admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:18.993485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:18.994090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:18.994749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:18.994910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:19.008029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:19.009514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:19.019375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1083/2146] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:19.303408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:19.304904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:19.317439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff976-0fbb-4fea-8e9f-11f2dcba449a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:19.323721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:19.324619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:19.324906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:19.325207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] PATH_INFO: `/v3/projects/5946c10adfb44f1398e02c2165f83bb0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:19.326443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5946c10adfb44f1398e02c2165f83bb0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:19.328483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:19.375290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5946c10adfb44f1398e02c2165f83bb0', 'request_id': 'req-3ac524a5-23e0-453c-96dc-7b4ea58948f5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.384115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5946c10adfb44f1398e02c2165f83bb0', 'request_id': 'req-3ac524a5-23e0-453c-96dc-7b4ea58948f5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.390730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5946c10adfb44f1398e02c2165f83bb0', 'request_id': 'req-3ac524a5-23e0-453c-96dc-7b4ea58948f5'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.401041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] The token cache is being invalidate because project 5946c10adfb44f1398e02c2165f83bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:19.401354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.402070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.403663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1064/2147] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:19 2022] DELETE /identity/v3/projects/5946c10adfb44f1398e02c2165f83bb0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:19.407926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:19.418447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726b765-3277-491f-b2a5-287e93cd3c6b service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:19.478049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:19.479071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:19.479359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:19.479631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] PATH_INFO: `/v3/users/360cf3ba97494f0e9764be2879121e09` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:19.480905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] RBAC: Authorizing `identity:delete_user(user_id=360cf3ba97494f0e9764be2879121e09)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:19.482923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:19.484417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:19.532601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360cf3ba97494f0e9764be2879121e09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.533193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360cf3ba97494f0e9764be2879121e09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.533784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360cf3ba97494f0e9764be2879121e09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.545982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360cf3ba97494f0e9764be2879121e09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.553003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-afcf151e-897a-4b7b-bcf3-0e4a3d6f843b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360cf3ba97494f0e9764be2879121e09'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:19.570960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1084/2148] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:19 2022] DELETE /identity/v3/users/360cf3ba97494f0e9764be2879121e09 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:19.852654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:19.854914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:19.869586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac524a5-23e0-453c-96dc-7b4ea58948f5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:19.889846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:19.938031 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:19.938914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:19.939195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:19.939472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] PATH_INFO: `/v3/projects/4bbf5df5f3934a9eb1186bfd19a6be11` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:19.940660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bbf5df5f3934a9eb1186bfd19a6be11)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:19.942612 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:19.996931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bbf5df5f3934a9eb1186bfd19a6be11', 'request_id': 'req-15ab21e8-ce89-4521-8d1b-6f39d3099e88'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:20.006285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bbf5df5f3934a9eb1186bfd19a6be11', 'request_id': 'req-15ab21e8-ce89-4521-8d1b-6f39d3099e88'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:20.012791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bbf5df5f3934a9eb1186bfd19a6be11', 'request_id': 'req-15ab21e8-ce89-4521-8d1b-6f39d3099e88'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:20.023722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] The token cache is being invalidate because project 4bbf5df5f3934a9eb1186bfd19a6be11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:20.024005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:20.024616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15ab21e8-ce89-4521-8d1b-6f39d3099e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:20.026281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1065/2149] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:19 2022] DELETE /identity/v3/projects/4bbf5df5f3934a9eb1186bfd19a6be11 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:20.032230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f5c5b-e153-4fc0-8a78-de1d326ebd30 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.032730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f5c5b-e153-4fc0-8a78-de1d326ebd30 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.032823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f5c5b-e153-4fc0-8a78-de1d326ebd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.057826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-af1f5c5b-e153-4fc0-8a78-de1d326ebd30 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:20.059230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-af1f5c5b-e153-4fc0-8a78-de1d326ebd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:20.126798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1085/2150] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:20.134850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590376-4ada-4aea-bdf1-53195f56888c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.135103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590376-4ada-4aea-bdf1-53195f56888c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.135515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590376-4ada-4aea-bdf1-53195f56888c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.155814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b5590376-4ada-4aea-bdf1-53195f56888c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:20.157099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b5590376-4ada-4aea-bdf1-53195f56888c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:20.165779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1066/2151] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:20.170720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f5c5b-e153-4fc0-8a78-de1d326ebd30 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:20.181082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f5c5b-e153-4fc0-8a78-de1d326ebd30 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:20.187658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:20.188348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.188713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.189075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.189953 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:15:20.190282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:20.192173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:20.205757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1086/2152] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:15:20.210206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5590376-4ada-4aea-bdf1-53195f56888c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:20.219809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5590376-4ada-4aea-bdf1-53195f56888c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:20.225947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:20.226604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.226883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.227155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.227837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:20.229784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:20.256582 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1067/2153] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:15:20 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:20.270460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:20.281310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95aa89-5608-4d46-8777-e2ea0151ea58 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:20.336697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:20.337397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.337676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.337952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.338739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:20.340634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:20.351125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:20.365114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1087/2154] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:15:20 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:20.366614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:20.377305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd94605-9167-4f10-a2c2-2713c49b681d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:20.381914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:20.382573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.382850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.383121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.383840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:20.385672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:20.400044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1068/2155] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:20.401689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:20.412313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e272d9-ff64-4e40-9309-9759ed42b836 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:20.418400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a2104-60f0-4563-b5e1-7f5a7fb86337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:20.419084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a2104-60f0-4563-b5e1-7f5a7fb86337 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.419358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a2104-60f0-4563-b5e1-7f5a7fb86337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.419732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a2104-60f0-4563-b5e1-7f5a7fb86337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.420338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a2104-60f0-4563-b5e1-7f5a7fb86337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:20.421999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a2104-60f0-4563-b5e1-7f5a7fb86337 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:20.436475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1088/2156] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:20.437654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:20.448606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834c34f-6800-4522-8fd9-e4b7d8c4fb80 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:20.453474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:20.454391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.454594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.454849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] PATH_INFO: `/v3/projects/2dd338e531914f218c91b1ccfc24adb3/users/e397f51850e14a419a5fb4e97047ea26/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.470808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd338e531914f218c91b1ccfc24adb3, user_id=e397f51850e14a419a5fb4e97047ea26, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:20.473471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:20.487773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1069/2157] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:20 2022] PUT /identity/v3/projects/2dd338e531914f218c91b1ccfc24adb3/users/e397f51850e14a419a5fb4e97047ea26/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:20.492694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51bea3-f493-4f07-8f2f-ca4abde34562 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.492986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51bea3-f493-4f07-8f2f-ca4abde34562 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.493305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51bea3-f493-4f07-8f2f-ca4abde34562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.511737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-7f51bea3-f493-4f07-8f2f-ca4abde34562 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:20.512937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7f51bea3-f493-4f07-8f2f-ca4abde34562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:20.575390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1089/2158] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:20.585164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:20.586944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:20.634137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:20.660860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5d459-aad8-4433-b1a6-5d9d84c90617 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:20.667216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:20.668106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:20.668480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:20.668990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:20.679785 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:20.681412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:20.692340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1070/2159] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:21.355669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b219f-a59a-43b2-81dd-2fefaa3db948 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:21.356286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b219f-a59a-43b2-81dd-2fefaa3db948 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:21.356404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b219f-a59a-43b2-81dd-2fefaa3db948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:21.408612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-767b219f-a59a-43b2-81dd-2fefaa3db948 None None] MFA Rules not processed for user `e397f51850e14a419a5fb4e97047ea26`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:21.411673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-767b219f-a59a-43b2-81dd-2fefaa3db948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:21.513423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1090/2160] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:21.514584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:21.525567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302f0e-fab3-4ba2-881f-ba63656495fc service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:21.532300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:21.533091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:21.533494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:21.533882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:21.534670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:21.536755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:21.574390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1071/2161] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:15:21 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:21.577249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b219f-a59a-43b2-81dd-2fefaa3db948 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:21.588601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b219f-a59a-43b2-81dd-2fefaa3db948 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:21.653323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:21.654355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:21.654636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:21.654965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:21.655779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:21.657862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:21.681299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1091/2162] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:15:21 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:21.686454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:21.697978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df71d8-093c-4802-81d4-b15177b7a76b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:21.704294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:21.704997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:21.705307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:21.705581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:21.706400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:21.708127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:21.729467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1072/2163] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:21.733969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:21.742332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:21.746630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7a5c-f6b8-436f-842b-54f6db6629c6 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:21.753507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:21.754725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:21.756835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:21.757401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] PATH_INFO: `/v3/projects/72f116188ac94f0aaadbc840e38153ca/users/28dd7c5f8be4437b94119be87b384992/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:21.780128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f116188ac94f0aaadbc840e38153ca, user_id=28dd7c5f8be4437b94119be87b384992, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:21.782807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:21.801097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1092/2164] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:21 2022] PUT /identity/v3/projects/72f116188ac94f0aaadbc840e38153ca/users/28dd7c5f8be4437b94119be87b384992/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:21.803546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:21.816519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7f9d2-2841-43ae-abb8-b81f40e3cfad admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:21.905575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe969f-2718-48a1-a0f7-95e9ba3b86e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:21.906260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe969f-2718-48a1-a0f7-95e9ba3b86e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:21.906566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe969f-2718-48a1-a0f7-95e9ba3b86e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:21.906830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe969f-2718-48a1-a0f7-95e9ba3b86e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:21.907642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe969f-2718-48a1-a0f7-95e9ba3b86e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:21.910247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe969f-2718-48a1-a0f7-95e9ba3b86e7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:21.925844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1073/2165] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:21.930144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:21.942467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab903dd-85d0-4c5d-be21-b20c609a804d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:21.950821 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:21.951917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:21.952247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:21.952630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] PATH_INFO: `/v3/projects/72f116188ac94f0aaadbc840e38153ca/users/28dd7c5f8be4437b94119be87b384992/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:21.954988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f116188ac94f0aaadbc840e38153ca, user_id=28dd7c5f8be4437b94119be87b384992, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:21.957840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:21.971364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1093/2166] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:21 2022] PUT /identity/v3/projects/72f116188ac94f0aaadbc840e38153ca/users/28dd7c5f8be4437b94119be87b384992/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:22.860936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ce82-2c6b-4288-a1b0-8d90de084d85 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:22.861540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ce82-2c6b-4288-a1b0-8d90de084d85 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:22.861540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ce82-2c6b-4288-a1b0-8d90de084d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:22.904279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d4b1ce82-2c6b-4288-a1b0-8d90de084d85 None None] MFA Rules not processed for user `28dd7c5f8be4437b94119be87b384992`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:22.905667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d4b1ce82-2c6b-4288-a1b0-8d90de084d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:22.992887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1074/2167] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:23.008819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:23.010714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:23.070402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19a762-9f4b-40af-a82f-df207798ae52 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:23.075334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0569-e1a4-4dd9-bc0d-27e09e766f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:23.076052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0569-e1a4-4dd9-bc0d-27e09e766f31 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:23.076334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0569-e1a4-4dd9-bc0d-27e09e766f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:23.076608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0569-e1a4-4dd9-bc0d-27e09e766f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:23.133495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0569-e1a4-4dd9-bc0d-27e09e766f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:23.135280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0569-e1a4-4dd9-bc0d-27e09e766f31 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:23.144905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1094/2168] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:24.540201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1ce82-2c6b-4288-a1b0-8d90de084d85 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:24.551439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1ce82-2c6b-4288-a1b0-8d90de084d85 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:24.555139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9779c75-e62e-42a4-bf69-94c3b47f63b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:24.555792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779c75-e62e-42a4-bf69-94c3b47f63b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:24.556076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779c75-e62e-42a4-bf69-94c3b47f63b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:24.556482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779c75-e62e-42a4-bf69-94c3b47f63b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:24.566734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9779c75-e62e-42a4-bf69-94c3b47f63b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:24.568924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9779c75-e62e-42a4-bf69-94c3b47f63b1 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:24.579163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1075/2169] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:33.140378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:34.203109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466eb815-de3a-4992-994b-9a2f04ac8860 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:34.204105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466eb815-de3a-4992-994b-9a2f04ac8860 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:34.204610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466eb815-de3a-4992-994b-9a2f04ac8860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:34.226615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-466eb815-de3a-4992-994b-9a2f04ac8860 None None] MFA Rules not processed for user `7093cc7cf9d1472db33b31e5f98a5141`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:34.227849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-466eb815-de3a-4992-994b-9a2f04ac8860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:34.312125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1095/2170] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:34.574572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:38.012448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b6c7c-9677-4be0-acbb-114fc1039074 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:38.013124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b6c7c-9677-4be0-acbb-114fc1039074 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:38.013124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b6c7c-9677-4be0-acbb-114fc1039074 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:38.014470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1076/2171] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:15:38.034421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f674-0ef5-4e56-95aa-fb01bcac760c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:38.034946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f674-0ef5-4e56-95aa-fb01bcac760c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:38.034946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f674-0ef5-4e56-95aa-fb01bcac760c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:38.054532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5490f674-0ef5-4e56-95aa-fb01bcac760c None None] MFA Rules not processed for user `60ca99d3679b4154a669661e28e8269b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:38.055792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5490f674-0ef5-4e56-95aa-fb01bcac760c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:38.132674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1096/2172] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:15:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:38.144761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b6c7c-9677-4be0-acbb-114fc1039074 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:38.147408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-895b6c7c-9677-4be0-acbb-114fc1039074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:38.165428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b6c7c-9677-4be0-acbb-114fc1039074 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:38.172006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:38.172651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:38.172934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:38.173239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:38.183194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:38.185109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:38.195683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1077/2173] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:38.879850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f674-0ef5-4e56-95aa-fb01bcac760c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:38.948908 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f674-0ef5-4e56-95aa-fb01bcac760c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:38.953884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:38.957314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:38.957314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:38.957314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] PATH_INFO: `/v3/users/e397f51850e14a419a5fb4e97047ea26` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:38.957314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e397f51850e14a419a5fb4e97047ea26)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:38.958636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:38.971387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:38.973137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:39.017533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:39.020581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e397f51850e14a419a5fb4e97047ea26'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.021600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e397f51850e14a419a5fb4e97047ea26'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.023118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e397f51850e14a419a5fb4e97047ea26'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.037326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e397f51850e14a419a5fb4e97047ea26'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.038939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550feb20-5a7d-4bea-937f-03e171705685 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:39.043276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:39.043991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:39.044372 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:39.044647 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:39.046270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e397f51850e14a419a5fb4e97047ea26'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.046459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:39.062494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1097/2174] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:38 2022] DELETE /identity/v3/users/e397f51850e14a419a5fb4e97047ea26 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:39.122147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:39.123602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:39.135689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1078/2175] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:39.306956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:39.308697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:39.392780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e29a2-0455-4d0b-bdcc-8b12f2fddbc1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:39.405658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:39.406555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:39.406853 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:39.407128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] PATH_INFO: `/v3/projects/2dd338e531914f218c91b1ccfc24adb3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:39.408316 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd338e531914f218c91b1ccfc24adb3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:39.410451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:39.463190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd338e531914f218c91b1ccfc24adb3', 'request_id': 'req-64478bec-e142-490c-bb57-c2236c28233e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.469771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd338e531914f218c91b1ccfc24adb3', 'request_id': 'req-64478bec-e142-490c-bb57-c2236c28233e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.477219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd338e531914f218c91b1ccfc24adb3', 'request_id': 'req-64478bec-e142-490c-bb57-c2236c28233e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.490924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] The token cache is being invalidate because project 2dd338e531914f218c91b1ccfc24adb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:39.491247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.491934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.493674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1098/2176] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:39 2022] DELETE /identity/v3/projects/2dd338e531914f218c91b1ccfc24adb3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:39.498237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:39.509010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf0977-b303-484d-904c-6d597ee873b8 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:39.569149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:39.569872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:39.570153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:39.570643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] PATH_INFO: `/v3/users/28dd7c5f8be4437b94119be87b384992` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:39.572243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=28dd7c5f8be4437b94119be87b384992)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:39.574540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:39.617376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:39.624497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28dd7c5f8be4437b94119be87b384992'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.625166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28dd7c5f8be4437b94119be87b384992'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.639304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28dd7c5f8be4437b94119be87b384992'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.640003 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28dd7c5f8be4437b94119be87b384992'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.646961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a0ded16-16d8-4bfe-9768-ffe4bb702dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28dd7c5f8be4437b94119be87b384992'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:39.662167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1079/2177] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:39 2022] DELETE /identity/v3/users/28dd7c5f8be4437b94119be87b384992 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:39.876129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:39.877916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:39.895811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64478bec-e142-490c-bb57-c2236c28233e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:39.981514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:39.992255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:39.993505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:39.994357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:39.994710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] PATH_INFO: `/v3/projects/72f116188ac94f0aaadbc840e38153ca` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:39.998363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f116188ac94f0aaadbc840e38153ca)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.000851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.063168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f116188ac94f0aaadbc840e38153ca', 'request_id': 'req-299be18c-3ab0-433e-a123-0be26625d572'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:40.070906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f116188ac94f0aaadbc840e38153ca', 'request_id': 'req-299be18c-3ab0-433e-a123-0be26625d572'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:40.079186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f116188ac94f0aaadbc840e38153ca', 'request_id': 'req-299be18c-3ab0-433e-a123-0be26625d572'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:40.093992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] The token cache is being invalidate because project 72f116188ac94f0aaadbc840e38153ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:40.094328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:40.095044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-299be18c-3ab0-433e-a123-0be26625d572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:40.096979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1099/2178] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:39 2022] DELETE /identity/v3/projects/72f116188ac94f0aaadbc840e38153ca => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:40.101989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957f62-8798-4b23-ab07-773a69471863 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.102477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957f62-8798-4b23-ab07-773a69471863 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.102570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957f62-8798-4b23-ab07-773a69471863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.121408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7d957f62-8798-4b23-ab07-773a69471863 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:40.123120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d957f62-8798-4b23-ab07-773a69471863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:40.193095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1080/2179] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:40.201824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eff1b-39a7-4890-a693-a0f9def1aa16 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.202220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eff1b-39a7-4890-a693-a0f9def1aa16 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.202564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eff1b-39a7-4890-a693-a0f9def1aa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.222386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ab3eff1b-39a7-4890-a693-a0f9def1aa16 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:40.223664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ab3eff1b-39a7-4890-a693-a0f9def1aa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:40.233402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1100/2180] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:40.237533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d957f62-8798-4b23-ab07-773a69471863 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:40.247325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d957f62-8798-4b23-ab07-773a69471863 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:40.252330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:40.256336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.259153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.259667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.260602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:15:40.260937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.263190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.277342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1081/2181] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:15:40.282008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eff1b-39a7-4890-a693-a0f9def1aa16 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:40.292384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eff1b-39a7-4890-a693-a0f9def1aa16 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:40.297504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:40.298306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.298678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.299014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.299793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.301815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.334862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1101/2182] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:15:40 2022] POST /identity/v3/projects => generated 359 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:40.335975 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:40.347060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd5b40-70a4-40a3-aef2-ef1b93830e5c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:40.413451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:40.414384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.415172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.415543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.417401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.419535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.452568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1082/2183] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:15:40 2022] POST /identity/v3/users => generated 280 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:40.457009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:40.473330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17a322-52a1-4862-8cce-26e72846114b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:40.490283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:40.491211 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.491669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.492196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.493169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.495611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.508593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1102/2184] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:40.512585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:40.535854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ccfa-ce17-45c3-9ae6-3e3d613c5c30 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:40.544530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d37247-9d13-4ad8-9bf2-a207c949623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:40.545490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d37247-9d13-4ad8-9bf2-a207c949623a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.545905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d37247-9d13-4ad8-9bf2-a207c949623a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.546345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d37247-9d13-4ad8-9bf2-a207c949623a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.547103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d37247-9d13-4ad8-9bf2-a207c949623a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.548698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d37247-9d13-4ad8-9bf2-a207c949623a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.559384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1083/2185] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:40.563660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:40.575444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5e49f-d715-44d2-942b-60661f481d2a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:40.581175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:40.582082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.582364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.582639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] PATH_INFO: `/v3/projects/c0e0bb678d864fc280a1248b0526a761/users/97fe27f5a0b24857837381dff697f18b/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.598497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e0bb678d864fc280a1248b0526a761, user_id=97fe27f5a0b24857837381dff697f18b, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.601605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.616437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1103/2186] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:40 2022] PUT /identity/v3/projects/c0e0bb678d864fc280a1248b0526a761/users/97fe27f5a0b24857837381dff697f18b/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:40.629033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9e27e-778d-481e-b092-a45e6dba53f8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.629490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9e27e-778d-481e-b092-a45e6dba53f8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.629843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9e27e-778d-481e-b092-a45e6dba53f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.658456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-56b9e27e-778d-481e-b092-a45e6dba53f8 None None] MFA Rules not processed for user `97fe27f5a0b24857837381dff697f18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:40.659679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56b9e27e-778d-481e-b092-a45e6dba53f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:40.669448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:40.726034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1084/2187] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:40 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:40.759108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:40.760967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:40.825402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4d4f-45ad-410c-a34b-b1d0690f21bc admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:40.831178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7161e-8930-4c7f-92af-8740652b9dd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:40.831952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7161e-8930-4c7f-92af-8740652b9dd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:40.832101 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7161e-8930-4c7f-92af-8740652b9dd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:40.832362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7161e-8930-4c7f-92af-8740652b9dd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:40.856579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7161e-8930-4c7f-92af-8740652b9dd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:40.858298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7161e-8930-4c7f-92af-8740652b9dd0 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:40.883939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1104/2188] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:40 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:40.895320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9e27e-778d-481e-b092-a45e6dba53f8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:41.056813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9e27e-778d-481e-b092-a45e6dba53f8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:41.065690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:41.070194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:41.070769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:41.071254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] PATH_INFO: `/v3/users/97fe27f5a0b24857837381dff697f18b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:41.075929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fe27f5a0b24857837381dff697f18b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:41.079964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:41.151616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fe27f5a0b24857837381dff697f18b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:41.153538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fe27f5a0b24857837381dff697f18b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:41.171176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fe27f5a0b24857837381dff697f18b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:41.172391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fe27f5a0b24857837381dff697f18b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:41.183331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fe27f5a0b24857837381dff697f18b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:41.201256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1085/2189] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:40 2022] DELETE /identity/v3/users/97fe27f5a0b24857837381dff697f18b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:41.205711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:41.206238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:41.206651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:41.227469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:41.228820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:41.245155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:41.351350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1105/2190] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:41.363989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:41.365614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:41.429848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:41.491993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b963f-b402-463b-9479-e2ee7efd9824 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:41.497033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48b02b-3156-41b0-9e0a-3f9d4df11c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:41.498044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48b02b-3156-41b0-9e0a-3f9d4df11c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:41.498324 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48b02b-3156-41b0-9e0a-3f9d4df11c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:41.498596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48b02b-3156-41b0-9e0a-3f9d4df11c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:41.509997 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48b02b-3156-41b0-9e0a-3f9d4df11c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:41.511434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48b02b-3156-41b0-9e0a-3f9d4df11c55 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:41.523008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1086/2191] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:42.144231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:42.146244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:42.173419 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a11af-c693-4ebc-81e1-6f4f7325b6d6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:42.178242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:42.181009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.181009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.181009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] PATH_INFO: `/v3/projects/c0e0bb678d864fc280a1248b0526a761` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.181330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0e0bb678d864fc280a1248b0526a761)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:42.184927 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:42.243835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0e0bb678d864fc280a1248b0526a761', 'request_id': 'req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:42.251253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0e0bb678d864fc280a1248b0526a761', 'request_id': 'req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:42.263156 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0e0bb678d864fc280a1248b0526a761', 'request_id': 'req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:42.282161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] The token cache is being invalidate because project c0e0bb678d864fc280a1248b0526a761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:42.282735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:42.283719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6304d6dc-4a09-4ca1-b8dd-1adac4473d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:42.286116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1106/2192] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:42 2022] DELETE /identity/v3/projects/c0e0bb678d864fc280a1248b0526a761 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:42.292233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03708256-d439-43f2-a0ef-76deee3cdc30 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.293367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03708256-d439-43f2-a0ef-76deee3cdc30 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.294246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03708256-d439-43f2-a0ef-76deee3cdc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.318827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-03708256-d439-43f2-a0ef-76deee3cdc30 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:42.321494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-03708256-d439-43f2-a0ef-76deee3cdc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:42.406816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1087/2193] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:42.414970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b801-e9fc-4949-9c93-79792c5c6c7b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.415256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b801-e9fc-4949-9c93-79792c5c6c7b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.416882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b801-e9fc-4949-9c93-79792c5c6c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.440344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-75c9b801-e9fc-4949-9c93-79792c5c6c7b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:42.441838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-75c9b801-e9fc-4949-9c93-79792c5c6c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:42.451352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1107/2194] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:42.456005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03708256-d439-43f2-a0ef-76deee3cdc30 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:42.470554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03708256-d439-43f2-a0ef-76deee3cdc30 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:42.476449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:42.477706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.478239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.478735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.479682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:15:42.480196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:42.482711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:42.502729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b801-e9fc-4949-9c93-79792c5c6c7b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:42.516829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1088/2195] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:15:42.517222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b801-e9fc-4949-9c93-79792c5c6c7b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:42.523433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd79ff-4aeb-4711-9595-d2e2dfff3d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:42.524504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd79ff-4aeb-4711-9595-d2e2dfff3d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.525158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd79ff-4aeb-4711-9595-d2e2dfff3d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.525949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd79ff-4aeb-4711-9595-d2e2dfff3d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.526892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd79ff-4aeb-4711-9595-d2e2dfff3d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:42.530029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd79ff-4aeb-4711-9595-d2e2dfff3d67 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:42.575849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1108/2196] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:15:42 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:42.579246 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:42.592376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b582a-a17a-4106-b6e8-beb1bc6fe39e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:42.723627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:42.724584 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.724968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.725627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.727670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:42.732791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:42.746823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015e96-53f8-40d1-87c9-8d233369415f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.747236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015e96-53f8-40d1-87c9-8d233369415f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.749216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015e96-53f8-40d1-87c9-8d233369415f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.782598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1089/2197] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:15:42 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:15:42.783914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:42.796022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11130fb-f3b6-499e-a036-326774c267b8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:42.801178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:42.802033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.802406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.802811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.803210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-10015e96-53f8-40d1-87c9-8d233369415f None None] MFA Rules not processed for user `ef6395fa14b54c43931ffeaab023d97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:42.803814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:42.805867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:42.806042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-10015e96-53f8-40d1-87c9-8d233369415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:42.822383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1090/2198] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:42.823727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:42.846673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924a53-c1e8-4466-9b2c-da78b89b5e72 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:42.852709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:42.853873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.854257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.854668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.856057 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:42.858359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:42.871602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1091/2199] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:15:42.879054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:42.885980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1109/2200] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:42.892904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036aaa-0061-4b5d-a550-aa5a5c75d900 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:42.898903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:42.900072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.900431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.900929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] PATH_INFO: `/v3/projects/9749d0c6191b4ed1b6174b6120c7c7d7/users/7cd1a5c4807b452ca36c309a47daf6b6/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.921428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9749d0c6191b4ed1b6174b6120c7c7d7, user_id=7cd1a5c4807b452ca36c309a47daf6b6, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:42.925344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:42.943047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1092/2201] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:15:42 2022] PUT /identity/v3/projects/9749d0c6191b4ed1b6174b6120c7c7d7/users/7cd1a5c4807b452ca36c309a47daf6b6/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:42.954138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607287ee-d54c-4c81-b91a-676dd4172758 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:42.954459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607287ee-d54c-4c81-b91a-676dd4172758 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:42.954701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607287ee-d54c-4c81-b91a-676dd4172758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:42.989102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-607287ee-d54c-4c81-b91a-676dd4172758 None None] MFA Rules not processed for user `7cd1a5c4807b452ca36c309a47daf6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:43.001761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-607287ee-d54c-4c81-b91a-676dd4172758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:43.014265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:43.098134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1110/2202] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:43.112983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:43.115129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:43.120816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:43.228887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385d03b-d262-4a33-9b14-10743540f55c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:43.234541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366a7e7596d14b32a6ce62790a2e3dd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:43.235366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:43.235746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:43.236120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:43.246757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:43.248390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:43.262830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1093/2203] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:47.095645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:47.096178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:47.096973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:47.119727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:47.120949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:47.201416 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1111/2204] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:15:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:47.264787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:47.267983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:47.371385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e8d7a-4b66-45ad-850c-a1b95e4ae8f7 service glance] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:47.377478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be77dd-240a-4150-904f-abd8bbc2e225 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:47.378369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be77dd-240a-4150-904f-abd8bbc2e225 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:47.378825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be77dd-240a-4150-904f-abd8bbc2e225 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:47.379366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be77dd-240a-4150-904f-abd8bbc2e225 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:47.389684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be77dd-240a-4150-904f-abd8bbc2e225 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:47.391687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be77dd-240a-4150-904f-abd8bbc2e225 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:47.405427 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1094/2205] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:15:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:15:49.641099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:49.642721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:49.836320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64948fc3-a77b-406f-ae71-da6361b2d435 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:49.854749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:49.855971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:49.856347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:49.856829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] PATH_INFO: `/v3/users/2cf79a5122aa4f5e846a8252e03e60c5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:49.858292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf79a5122aa4f5e846a8252e03e60c5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:49.861588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:49.935605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf79a5122aa4f5e846a8252e03e60c5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:49.936440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf79a5122aa4f5e846a8252e03e60c5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:49.937042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf79a5122aa4f5e846a8252e03e60c5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:49.973933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:49.990690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf79a5122aa4f5e846a8252e03e60c5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:50.001777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf79a5122aa4f5e846a8252e03e60c5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:50.038010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:50.038648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:50.038744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1112/2206] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:15:49 2022] DELETE /identity/v3/users/2cf79a5122aa4f5e846a8252e03e60c5 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:15:50.038833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:50.094050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:15:50.095940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:50.243882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1095/2207] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:15:50.252097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:50.263259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:50.438514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01cbb8-92bd-46b6-9e21-9dc28d1878cd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:50.456577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33258a-9df8-4350-8d2f-511b7cb6432b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:50.457583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33258a-9df8-4350-8d2f-511b7cb6432b service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:50.458235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33258a-9df8-4350-8d2f-511b7cb6432b service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:50.458727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33258a-9df8-4350-8d2f-511b7cb6432b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:50.473537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33258a-9df8-4350-8d2f-511b7cb6432b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:50.475266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33258a-9df8-4350-8d2f-511b7cb6432b service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:50.506319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1113/2208] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:15:51.170623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:15:51.172742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:15:51.205908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71b43-62bb-43ae-8336-88a4fb42c286 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:15:51.219597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:15:51.220654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:15:51.220825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:15:51.221137 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] PATH_INFO: `/v3/projects/2b71b7fd73dc4c8892afbc68ae2b99e9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:15:51.223130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b71b7fd73dc4c8892afbc68ae2b99e9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:15:51.225016 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:15:51.241563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:15:51.293265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b71b7fd73dc4c8892afbc68ae2b99e9', 'request_id': 'req-8558aba0-22be-4324-80ef-25414ffc0e6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:51.303350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b71b7fd73dc4c8892afbc68ae2b99e9', 'request_id': 'req-8558aba0-22be-4324-80ef-25414ffc0e6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:51.312252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b71b7fd73dc4c8892afbc68ae2b99e9', 'request_id': 'req-8558aba0-22be-4324-80ef-25414ffc0e6c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:51.324287 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] The token cache is being invalidate because project 2b71b7fd73dc4c8892afbc68ae2b99e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:15:51.324717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:51.325729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8558aba0-22be-4324-80ef-25414ffc0e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:15:51.329250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1096/2209] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:15:51 2022] DELETE /identity/v3/projects/2b71b7fd73dc4c8892afbc68ae2b99e9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:16:00.484100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=35,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:16:01.320739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:16:04.979799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce17baf-bfb0-4e03-960d-96d51a64749a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:04.980413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce17baf-bfb0-4e03-960d-96d51a64749a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:04.980526 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce17baf-bfb0-4e03-960d-96d51a64749a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:04.982071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1114/2210] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:16:04.987834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268bc41-3748-408a-92da-7075735a9287 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:04.988315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268bc41-3748-408a-92da-7075735a9287 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:04.988416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268bc41-3748-408a-92da-7075735a9287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.012561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3268bc41-3748-408a-92da-7075735a9287 None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:16:05.013827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3268bc41-3748-408a-92da-7075735a9287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:05.095656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1097/2211] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:16:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:16:05.107881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce17baf-bfb0-4e03-960d-96d51a64749a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:05.109618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6ce17baf-bfb0-4e03-960d-96d51a64749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:05.176005 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce17baf-bfb0-4e03-960d-96d51a64749a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:05.179622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3bfb0-03d9-4188-a8ef-35ea0fdd5c22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366a7e7596d14b32a6ce62790a2e3dd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:05.180270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3bfb0-03d9-4188-a8ef-35ea0fdd5c22 service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:05.180549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3bfb0-03d9-4188-a8ef-35ea0fdd5c22 service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:05.180819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3bfb0-03d9-4188-a8ef-35ea0fdd5c22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.191141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3bfb0-03d9-4188-a8ef-35ea0fdd5c22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:05.192587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3bfb0-03d9-4188-a8ef-35ea0fdd5c22 service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:05.203428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1115/2212] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:16:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:16:05.272416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8db3-0005-4e7e-9ba4-559517dec71e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:05.272704 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8db3-0005-4e7e-9ba4-559517dec71e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:05.272982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8db3-0005-4e7e-9ba4-559517dec71e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.274414 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1098/2213] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:16:05.279286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188d967-f4fe-4871-b27e-e6290e106e7a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:05.279597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188d967-f4fe-4871-b27e-e6290e106e7a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:05.279878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188d967-f4fe-4871-b27e-e6290e106e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.299605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a188d967-f4fe-4871-b27e-e6290e106e7a None None] MFA Rules not processed for user `754c137a554941d99f7ef9c380a55ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:16:05.300816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a188d967-f4fe-4871-b27e-e6290e106e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:05.353020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1116/2214] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:16:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:16:05.365166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d8db3-0005-4e7e-9ba4-559517dec71e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:05.367105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e37d8db3-0005-4e7e-9ba4-559517dec71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:05.381233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d8db3-0005-4e7e-9ba4-559517dec71e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:05.385078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417a690-56ff-41c4-b2d8-5ad12474c650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:05.385714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417a690-56ff-41c4-b2d8-5ad12474c650 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:05.385995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417a690-56ff-41c4-b2d8-5ad12474c650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:05.386269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417a690-56ff-41c4-b2d8-5ad12474c650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.393828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417a690-56ff-41c4-b2d8-5ad12474c650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:05.395291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417a690-56ff-41c4-b2d8-5ad12474c650 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:05.403379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1099/2215] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:16:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:16:05.489559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd419ce-fd53-4faf-b7ef-bb31fb8f8a1a None None] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:05.489852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd419ce-fd53-4faf-b7ef-bb31fb8f8a1a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:05.490159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd419ce-fd53-4faf-b7ef-bb31fb8f8a1a None None] PATH_INFO: `/` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.491600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1117/2216] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 07:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 07:16:05.496833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:05.497152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:05.497425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.516689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] MFA Rules not processed for user `95c860dcbf3b40bbb7427a417357c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:16:05.517888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:05.528639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1100/2217] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 07:16:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:16:05.540914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd419ce-fd53-4faf-b7ef-bb31fb8f8a1a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:05.554346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd419ce-fd53-4faf-b7ef-bb31fb8f8a1a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:05.560045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe191993-1499-45a6-89b7-f916bf5a95c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:05.560705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe191993-1499-45a6-89b7-f916bf5a95c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:05.561041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe191993-1499-45a6-89b7-f916bf5a95c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:05.561286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe191993-1499-45a6-89b7-f916bf5a95c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:05.575240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe191993-1499-45a6-89b7-f916bf5a95c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:05.576692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe191993-1499-45a6-89b7-f916bf5a95c8 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:05.587859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1118/2218] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:16:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:16:09.863314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:09.865458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:09.939453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa429e2-4f43-4c81-a169-3e50edbf513d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:09.945777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:09.946445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:09.946732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:09.947010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:09.948794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:09.961566 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:09.963121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:09.974870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1101/2219] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:16:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:16:12.288040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:12.288532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:12.288770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:12.308434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:16:12.309672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:12.365386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1119/2220] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:16:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:16:12.377579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:12.379061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:12.445736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:16:12.450455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fa5fb-e2ec-47fe-801a-52c10cf27601 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:12.456788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937578a-bafd-4c21-ad20-841f85d98625 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:12.457492 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937578a-bafd-4c21-ad20-841f85d98625 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:12.457769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937578a-bafd-4c21-ad20-841f85d98625 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:12.458039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937578a-bafd-4c21-ad20-841f85d98625 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:12.467157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937578a-bafd-4c21-ad20-841f85d98625 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:12.468769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937578a-bafd-4c21-ad20-841f85d98625 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:12.477332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1102/2221] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:16:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:16:14.951840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:14.953664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:15.010781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:16:15.036710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef290de0-9367-4093-9fd3-d046f75e559b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:15.042125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:15.043281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:15.043281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:15.043572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] PATH_INFO: `/v3/users/7cd1a5c4807b452ca36c309a47daf6b6` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:15.045100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd1a5c4807b452ca36c309a47daf6b6)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:15.046856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:15.099201 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd1a5c4807b452ca36c309a47daf6b6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.100252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd1a5c4807b452ca36c309a47daf6b6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.101221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd1a5c4807b452ca36c309a47daf6b6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.114324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd1a5c4807b452ca36c309a47daf6b6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.121433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd1a5c4807b452ca36c309a47daf6b6'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.135383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1120/2222] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:16:14 2022] DELETE /identity/v3/users/7cd1a5c4807b452ca36c309a47daf6b6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:16:15.140070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:15.140648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:15.140648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:15.160592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:16:15.162159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:15.242462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1103/2223] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:16:15.251813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:15.253471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:15.314216 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d7cfc-4e19-4a8d-9862-ccef3a63b828 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:15.318234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44430436-f70c-4da6-ad73-5b47975c8fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:15.318890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44430436-f70c-4da6-ad73-5b47975c8fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:15.319176 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44430436-f70c-4da6-ad73-5b47975c8fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:15.319455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44430436-f70c-4da6-ad73-5b47975c8fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:15.329705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44430436-f70c-4da6-ad73-5b47975c8fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:15.331669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44430436-f70c-4da6-ad73-5b47975c8fa0 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:15.342099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1121/2224] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:16:15.651322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:15.652773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:15.666038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e710dc-3dd8-4744-8580-87b3212e356e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:15.684359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:15.685396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:15.685806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:15.686141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] PATH_INFO: `/v3/projects/9749d0c6191b4ed1b6174b6120c7c7d7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:15.688780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9749d0c6191b4ed1b6174b6120c7c7d7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:15.691165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:15.770793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9749d0c6191b4ed1b6174b6120c7c7d7', 'request_id': 'req-d3596afb-29c6-447b-998e-f627a3033224'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.785029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9749d0c6191b4ed1b6174b6120c7c7d7', 'request_id': 'req-d3596afb-29c6-447b-998e-f627a3033224'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.800224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9749d0c6191b4ed1b6174b6120c7c7d7', 'request_id': 'req-d3596afb-29c6-447b-998e-f627a3033224'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.819883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] The token cache is being invalidate because project 9749d0c6191b4ed1b6174b6120c7c7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:16:15.820201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.820992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:16:15.823249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1104/2225] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:16:15 2022] DELETE /identity/v3/projects/9749d0c6191b4ed1b6174b6120c7c7d7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:16:25.337906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:16:25.813425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:16:40.754214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cad02b-3712-425f-aa1f-90642927417b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:40.754745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cad02b-3712-425f-aa1f-90642927417b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:40.754745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cad02b-3712-425f-aa1f-90642927417b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:40.782597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-a3cad02b-3712-425f-aa1f-90642927417b None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:16:40.783810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a3cad02b-3712-425f-aa1f-90642927417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:40.844704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1122/2226] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 07:16:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:16:40.857880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:16:40.859498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:16:40.931944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3596afb-29c6-447b-998e-f627a3033224 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:16:40.937029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec887328-2490-4955-9bc1-979e51f67224 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:16:40.937715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec887328-2490-4955-9bc1-979e51f67224 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:16:40.937995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec887328-2490-4955-9bc1-979e51f67224 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:16:40.938267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec887328-2490-4955-9bc1-979e51f67224 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:16:40.947060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec887328-2490-4955-9bc1-979e51f67224 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:16:40.948685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec887328-2490-4955-9bc1-979e51f67224 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:16:40.956886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1105/2227] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:16:50.838108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=25 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:16:50.953847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=32 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:17:17.105237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb028-2eb4-4e09-b898-011c65bb7f47 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:17:17.105802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb028-2eb4-4e09-b898-011c65bb7f47 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:17:17.105802 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb028-2eb4-4e09-b898-011c65bb7f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:17:17.125487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1adcb028-2eb4-4e09-b898-011c65bb7f47 None None] MFA Rules not processed for user `ef6395fa14b54c43931ffeaab023d97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:17:17.126863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1adcb028-2eb4-4e09-b898-011c65bb7f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:17:17.184420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1123/2228] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:17:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:17:27.177175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=25 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:17:29.089245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0aa9-8e89-4aed-9958-4c19227e669c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:17:29.089695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0aa9-8e89-4aed-9958-4c19227e669c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:17:29.089916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0aa9-8e89-4aed-9958-4c19227e669c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:17:29.109509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-acfb0aa9-8e89-4aed-9958-4c19227e669c None None] MFA Rules not processed for user `7093cc7cf9d1472db33b31e5f98a5141`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:17:29.110850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-acfb0aa9-8e89-4aed-9958-4c19227e669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:17:29.177444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1106/2229] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:17:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:17:32.359188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:17:32.359746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:17:32.359964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:17:32.378808 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] MFA Rules not processed for user `53278c1c534344978ca40d7e2313c02c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:17:32.380210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:17:32.438767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1124/2230] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:17:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:17:39.164864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=25 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:17:42.431518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=25 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:17:49.527356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422e55-6274-4836-adde-fc3778380d00 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:17:49.527859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422e55-6274-4836-adde-fc3778380d00 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:17:49.527973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422e55-6274-4836-adde-fc3778380d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:17:49.546077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e1422e55-6274-4836-adde-fc3778380d00 None None] MFA Rules not processed for user `583143af14a645238262214515ee148f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:17:49.547260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e1422e55-6274-4836-adde-fc3778380d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:17:49.604147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1107/2231] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:17:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:17:59.596660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=25 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:17:59.930026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:17:59.931616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:17:59.990116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9116d69-9283-4b97-8214-e7f09ae5a1c2 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:17:59.994317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:17:59.994747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:17:59.995060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:17:59.995303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:17:59.997150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:00.074982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:00.076436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:00.087726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1125/2232] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:17:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:01.284663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422e55-6274-4836-adde-fc3778380d00 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:01.295992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422e55-6274-4836-adde-fc3778380d00 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:01.300522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:01.301436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:01.301852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:01.302130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:01.304229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:01.318776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:01.320254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:01.331406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1108/2233] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:10.082867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=54 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:11.326626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:18.368059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:18.369861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:18.384069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197a5eb-3710-491b-ad2b-e095af060f57 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:18.391271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:18.391890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:18.392190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:18.392456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:18.394538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:18.405692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:18.407291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:18.416870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1126/2234] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:21.229674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:21.231175 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:21.290519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33735c92-17af-4672-b2a5-eed574283f69 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:21.294828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:21.295486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:21.295733 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:21.296033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:21.298200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:21.309683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:21.311100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:21.320369 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1109/2235] 10.222.0.25 () {60 vars in 1396 bytes} [Tue Jul 12 07:18:21 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:23.435109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:23.437335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:23.516099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264f228-8644-48e9-b1f0-8837bba78694 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:23.522151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:23.522884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:23.523168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:23.523437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:23.526013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:23.541104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:23.542692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:23.555506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1127/2236] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:31.316344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=26 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:31.882250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:31.893518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83773d6-8826-4093-beb2-1489eaebe1bd service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:31.897885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:31.898544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:31.898831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:31.899108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:31.900984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:31.911808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:31.913282 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:31.921534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1110/2237] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:33.551240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=38 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:33.638163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:33.654553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d604-919a-424c-bbfa-29d0bc6b314d service placement] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:33.663817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:33.664771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:33.665222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:33.665636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:33.668305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:33.684593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:33.686285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:33.698185 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1128/2238] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:39.970439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:39.984093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a1af3-dc17-453a-9987-b575d2daed3d service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:39.996222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31eabe-a0d4-4ccb-a3c1-a156af33bff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:39.997245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31eabe-a0d4-4ccb-a3c1-a156af33bff1 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:39.998598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31eabe-a0d4-4ccb-a3c1-a156af33bff1 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:39.998879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31eabe-a0d4-4ccb-a3c1-a156af33bff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:40.000764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ed31eabe-a0d4-4ccb-a3c1-a156af33bff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:40.016291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31eabe-a0d4-4ccb-a3c1-a156af33bff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:40.017991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31eabe-a0d4-4ccb-a3c1-a156af33bff1 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:40.029614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1111/2239] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:41.044636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:41.054637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07aea1-ff62-411a-af04-87f9c98ad246 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:41.058392 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:41.059114 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:41.059481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:41.059819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:41.061660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:41.130420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:41.131898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:41.143271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1129/2240] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:44.207540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48008-6a06-4bfe-ac25-687f2ff214e0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:44.208104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48008-6a06-4bfe-ac25-687f2ff214e0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:44.208341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48008-6a06-4bfe-ac25-687f2ff214e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:44.228817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d4a48008-6a06-4bfe-ac25-687f2ff214e0 None None] MFA Rules not processed for user `ef6395fa14b54c43931ffeaab023d97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:18:44.230064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d4a48008-6a06-4bfe-ac25-687f2ff214e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:44.238876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1112/2241] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:18:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:18:44.874215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:44.883226 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89920c9b-790c-4b54-86ab-947ea4db8212 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:44.887129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:44.887815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:44.888099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:44.888382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:44.890222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:44.901991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:44.903792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:44.912774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1130/2242] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:45.671528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a48008-6a06-4bfe-ac25-687f2ff214e0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:45.686157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a48008-6a06-4bfe-ac25-687f2ff214e0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:45.693546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:45.694295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:45.694664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:45.694930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:45.697091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:45.712378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:45.714392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:45.733400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1113/2243] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:45.742276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:45.743872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:45.764651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1331fd-79fd-4a30-8145-ed7e152ed656 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:45.770281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:45.771274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:45.771631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:45.771970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] PATH_INFO: `/v3/users/7093cc7cf9d1472db33b31e5f98a5141` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:45.773480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7093cc7cf9d1472db33b31e5f98a5141)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:45.776188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:45.830388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7093cc7cf9d1472db33b31e5f98a5141'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:45.831164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7093cc7cf9d1472db33b31e5f98a5141'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:45.831765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7093cc7cf9d1472db33b31e5f98a5141'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:45.846424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7093cc7cf9d1472db33b31e5f98a5141'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:45.854562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9ca46da3-71ae-4366-906a-b73b0a78e88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7093cc7cf9d1472db33b31e5f98a5141'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:45.867799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1131/2244] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:18:45 2022] DELETE /identity/v3/users/7093cc7cf9d1472db33b31e5f98a5141 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:18:46.067000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.078559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce0dc0-e23d-4f88-a58e-f4e2fce563ec service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.136600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.137437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.137718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.137988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] PATH_INFO: `/v3/projects/02510f7812444c59bec582ff1127aa13` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.139319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=02510f7812444c59bec582ff1127aa13)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.141257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.194879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02510f7812444c59bec582ff1127aa13', 'request_id': 'req-252f857b-a344-4edb-b3f5-b58dfeff0dda'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:46.203846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02510f7812444c59bec582ff1127aa13', 'request_id': 'req-252f857b-a344-4edb-b3f5-b58dfeff0dda'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:46.210197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02510f7812444c59bec582ff1127aa13', 'request_id': 'req-252f857b-a344-4edb-b3f5-b58dfeff0dda'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:46.220399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] The token cache is being invalidate because project 02510f7812444c59bec582ff1127aa13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:18:46.220686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:46.221482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-252f857b-a344-4edb-b3f5-b58dfeff0dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:18:46.223029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1114/2245] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:18:46 2022] DELETE /identity/v3/projects/02510f7812444c59bec582ff1127aa13 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:18:46.228121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb9189-0c66-4ad2-a6cf-d9195fee592a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.228591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb9189-0c66-4ad2-a6cf-d9195fee592a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.228684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb9189-0c66-4ad2-a6cf-d9195fee592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.247131 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e9bb9189-0c66-4ad2-a6cf-d9195fee592a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:18:46.248335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e9bb9189-0c66-4ad2-a6cf-d9195fee592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:46.289704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:46.318247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1132/2246] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:18:46.326271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4efe17-7df0-4956-b8da-56c58afcfbf9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.326571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4efe17-7df0-4956-b8da-56c58afcfbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.326848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4efe17-7df0-4956-b8da-56c58afcfbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.347160 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7e4efe17-7df0-4956-b8da-56c58afcfbf9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:18:46.348400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7e4efe17-7df0-4956-b8da-56c58afcfbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:46.357929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1115/2247] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:18:46.362412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb9189-0c66-4ad2-a6cf-d9195fee592a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.372572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb9189-0c66-4ad2-a6cf-d9195fee592a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.377711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.378758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.379100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.379321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.380053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:18:46.380362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.382040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.395346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1133/2248] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:18:46.399767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4efe17-7df0-4956-b8da-56c58afcfbf9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.410792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4efe17-7df0-4956-b8da-56c58afcfbf9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.415654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.416476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.416762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.417124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.417790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.419863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.448869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1116/2249] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:18:46.453220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.463788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af569f0-2e1b-42ea-9ff0-0b82a369a6e7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.552665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.553623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.553910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.554188 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.554992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.557341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.580027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1134/2250] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:18:46 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:18:46.584882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.595709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2235083-bf8b-4a19-bc00-9811da9924fc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.601428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.602342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.602498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.602729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.603386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.605242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.620825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1117/2251] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:18:46.623495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.633777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d499f5-b5db-46fa-b6ec-e12ec8dccda8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.638402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b94cc-effe-4176-8fcb-f9e59fbd094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.639076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b94cc-effe-4176-8fcb-f9e59fbd094f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.639358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b94cc-effe-4176-8fcb-f9e59fbd094f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.639634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b94cc-effe-4176-8fcb-f9e59fbd094f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.640368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b94cc-effe-4176-8fcb-f9e59fbd094f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.642019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b94cc-effe-4176-8fcb-f9e59fbd094f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.653772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1135/2252] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:18:46.657721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.664655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:46.668685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143947-3e89-4663-8093-6f2eccbe2420 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.675220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.676147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.676631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.677011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] PATH_INFO: `/v3/projects/4bc4c559ab454a6684c5aabacdee3e2f/users/2c7ebeabae6d4a03a2457122a87078c1/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.693190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc4c559ab454a6684c5aabacdee3e2f, user_id=2c7ebeabae6d4a03a2457122a87078c1, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.695821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.713135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1118/2253] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:18:46 2022] PUT /identity/v3/projects/4bc4c559ab454a6684c5aabacdee3e2f/users/2c7ebeabae6d4a03a2457122a87078c1/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:18:46.724147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.724441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.724745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.752813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] MFA Rules not processed for user `2c7ebeabae6d4a03a2457122a87078c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:18:46.754487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:46.819847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1136/2254] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:18:46.824398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.891364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb326da-1bef-46a6-87f8-b9814f3910c9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:46.896326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:46.896999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:46.897323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:46.897596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:46.898286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:46.900518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:46.911168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.912996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:46.934311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1119/2255] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:18:46.937659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:46.963018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:47.009196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d4e39-ddca-4e67-9d53-1b8f4892742a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.027715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34e93-7d8c-41fb-ba26-b6dffc46f7e1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.032755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.033901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.034185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.034459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.035166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.037111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.059713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1120/2256] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:18:46 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:18:47.064309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:47.074580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c69772-7240-47d8-8de5-385101883798 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.079962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.081276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.081276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.081519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.081641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.082106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.082422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.082797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.083189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.083718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.085364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:47.095942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1121/2257] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:18:47.100239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:47.110675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928e26-17ca-48c3-a538-74513d4059c5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.117141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.118050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.118336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.118615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] PATH_INFO: `/v3/projects/fff3e5aedee14e2d8ff92a28dae5b963/users/042725f03fc94949927a2e3d21dde202/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.140149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff3e5aedee14e2d8ff92a28dae5b963, user_id=042725f03fc94949927a2e3d21dde202, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.143437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.163425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.166276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.168404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1122/2258] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:18:47 2022] PUT /identity/v3/projects/fff3e5aedee14e2d8ff92a28dae5b963/users/042725f03fc94949927a2e3d21dde202/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:18:47.172018 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:47.179639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1137/2259] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:18:47.214030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:47.276202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f77b20-ee25-4661-a7e6-fa944430a077 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.280964 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d8c6b-2aa7-463f-a311-9c55cb9778cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.281675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d8c6b-2aa7-463f-a311-9c55cb9778cf admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.281965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d8c6b-2aa7-463f-a311-9c55cb9778cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.282232 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d8c6b-2aa7-463f-a311-9c55cb9778cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.283340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d8c6b-2aa7-463f-a311-9c55cb9778cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.285112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d8c6b-2aa7-463f-a311-9c55cb9778cf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.297129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1123/2260] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:18:47.301481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:47.326284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce56f7-c70e-4fc2-8cbd-352461215e3d service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.332523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.333487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.333766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.334052 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] PATH_INFO: `/v3/projects/fff3e5aedee14e2d8ff92a28dae5b963/users/042725f03fc94949927a2e3d21dde202/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.336088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff3e5aedee14e2d8ff92a28dae5b963, user_id=042725f03fc94949927a2e3d21dde202, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.338743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.353948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1138/2261] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:18:47 2022] PUT /identity/v3/projects/fff3e5aedee14e2d8ff92a28dae5b963/users/042725f03fc94949927a2e3d21dde202/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:18:47.364435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a78c-720a-46c3-adeb-4235ca322cff None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.364764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a78c-720a-46c3-adeb-4235ca322cff None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.365139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a78c-720a-46c3-adeb-4235ca322cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.396784 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f4b9a78c-720a-46c3-adeb-4235ca322cff None None] MFA Rules not processed for user `042725f03fc94949927a2e3d21dde202`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:18:47.398190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f4b9a78c-720a-46c3-adeb-4235ca322cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:47.493118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1124/2262] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:18:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:18:47.500730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:47.505652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:47.615624 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3113c35-e0c0-4d00-b69d-a5d834700824 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.620322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.620924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.621245 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.621524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.634701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.636352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.642702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:47.648391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1139/2263] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:47.730830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9a78c-720a-46c3-adeb-4235ca322cff None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:47.741011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9a78c-720a-46c3-adeb-4235ca322cff None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:47.746311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a74d84-6093-4dea-8b28-c0fc38531014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:47.747102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a74d84-6093-4dea-8b28-c0fc38531014 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:47.747391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a74d84-6093-4dea-8b28-c0fc38531014 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:47.747668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a74d84-6093-4dea-8b28-c0fc38531014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:47.830510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a74d84-6093-4dea-8b28-c0fc38531014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:47.832010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a74d84-6093-4dea-8b28-c0fc38531014 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:47.845459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1125/2264] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:18:55.115918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:55.115918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:55.190921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd245a1-61fe-4609-b417-a7174e5fa79b service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:55.196756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:55.197462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:55.197743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:55.198018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:55.199837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:55.212026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:55.213681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:55.223967 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1140/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:18:57.837907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=82 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:18:58.090045 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ee944-c464-4be6-a9e3-6618fc3ecd3a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:58.090306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ee944-c464-4be6-a9e3-6618fc3ecd3a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:58.090583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ee944-c464-4be6-a9e3-6618fc3ecd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:58.110509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2b1ee944-c464-4be6-a9e3-6618fc3ecd3a None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:18:58.111780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2b1ee944-c464-4be6-a9e3-6618fc3ecd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:58.185264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1126/2266] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:18:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:18:58.245635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:58.247115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:58.327405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f9fa0-89c4-4142-a2ed-8866c2db0a8d service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:58.333411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:58.334073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:58.334355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:58.334627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:58.343861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:58.345502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:58.367653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1141/2267] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:18:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:18:58.815855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ee944-c464-4be6-a9e3-6618fc3ecd3a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:18:58.832426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ee944-c464-4be6-a9e3-6618fc3ecd3a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:18:58.856134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49762968-918e-4f9b-84e4-4222c6e8f36b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:18:58.856871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762968-918e-4f9b-84e4-4222c6e8f36b service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:18:58.857136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762968-918e-4f9b-84e4-4222c6e8f36b service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:18:58.857820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762968-918e-4f9b-84e4-4222c6e8f36b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:18:58.859717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-49762968-918e-4f9b-84e4-4222c6e8f36b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:18:58.877503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49762968-918e-4f9b-84e4-4222c6e8f36b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:18:58.879172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49762968-918e-4f9b-84e4-4222c6e8f36b service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:18:58.904089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1127/2268] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:18:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:08.351794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=66 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:08.897882 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=31 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:18.871725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:18.873425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:18.931315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f142c9-9bd7-4209-b945-bf91a438ccb8 service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:18.935087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:18.935736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:18.936013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:18.936282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:18.938220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:18.948229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:18.949676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:18.958275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1142/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:25.638982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cedf7-eb22-4c52-9e76-21c49fe0578e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:25.639548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cedf7-eb22-4c52-9e76-21c49fe0578e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:25.639548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cedf7-eb22-4c52-9e76-21c49fe0578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:25.659059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-816cedf7-eb22-4c52-9e76-21c49fe0578e None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:25.660289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-816cedf7-eb22-4c52-9e76-21c49fe0578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:25.667955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1128/2270] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:19:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:25.680027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:25.693590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af20ab4-ccd2-42cc-b612-3575333659c7 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:25.701783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1703fac-7b0d-45ca-92c4-7a608cc437a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:25.702873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703fac-7b0d-45ca-92c4-7a608cc437a9 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:25.703201 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703fac-7b0d-45ca-92c4-7a608cc437a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:25.703645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703fac-7b0d-45ca-92c4-7a608cc437a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:25.715736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1703fac-7b0d-45ca-92c4-7a608cc437a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:25.717531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1703fac-7b0d-45ca-92c4-7a608cc437a9 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:25.726772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1143/2271] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:19:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:19:29.392310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cedf7-eb22-4c52-9e76-21c49fe0578e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:29.460574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cedf7-eb22-4c52-9e76-21c49fe0578e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:29.465972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:29.466995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:29.467275 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:29.467552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] PATH_INFO: `/v3/users/2c7ebeabae6d4a03a2457122a87078c1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:29.468756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7ebeabae6d4a03a2457122a87078c1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:29.470769 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:29.529673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7ebeabae6d4a03a2457122a87078c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:29.530640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7ebeabae6d4a03a2457122a87078c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:29.546467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7ebeabae6d4a03a2457122a87078c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:29.547771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7ebeabae6d4a03a2457122a87078c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:29.562518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7ebeabae6d4a03a2457122a87078c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:29.588469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1129/2272] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:29 2022] DELETE /identity/v3/users/2c7ebeabae6d4a03a2457122a87078c1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:29.599488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:29.599950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:29.600039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:29.629029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:29.630358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:29.705089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1144/2273] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:29.713716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:29.715458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:29.803376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04f8b7-a351-497d-a945-9eef44fa8522 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:29.807156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:29.807844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:29.808126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:29.808401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:29.818364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:29.820731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:29.832594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1130/2274] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:30.149789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:30.151270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:30.165884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f6620-6419-4b6b-9eb7-37c4024afa14 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:30.171962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:30.173264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:30.173876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:30.174320 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] PATH_INFO: `/v3/projects/4bc4c559ab454a6684c5aabacdee3e2f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:30.176488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc4c559ab454a6684c5aabacdee3e2f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:30.178462 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:30.237708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc4c559ab454a6684c5aabacdee3e2f', 'request_id': 'req-1e774561-cd81-4f60-89af-7917f833a785'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.245969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc4c559ab454a6684c5aabacdee3e2f', 'request_id': 'req-1e774561-cd81-4f60-89af-7917f833a785'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.252396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc4c559ab454a6684c5aabacdee3e2f', 'request_id': 'req-1e774561-cd81-4f60-89af-7917f833a785'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.268667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] The token cache is being invalidate because project 4bc4c559ab454a6684c5aabacdee3e2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:30.268958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.269674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.271429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1145/2275] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:30 2022] DELETE /identity/v3/projects/4bc4c559ab454a6684c5aabacdee3e2f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:30.275424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:30.286157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78588b80-b209-4181-aab8-7f3999a9bed3 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:30.298991 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:30.375990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:30.376886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:30.377199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:30.377473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] PATH_INFO: `/v3/users/042725f03fc94949927a2e3d21dde202` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:30.378677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042725f03fc94949927a2e3d21dde202)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:30.382613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:30.451220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042725f03fc94949927a2e3d21dde202'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.452105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042725f03fc94949927a2e3d21dde202'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.466800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042725f03fc94949927a2e3d21dde202'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.467531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042725f03fc94949927a2e3d21dde202'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.476162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042725f03fc94949927a2e3d21dde202'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:30.497334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1131/2276] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:30 2022] DELETE /identity/v3/users/042725f03fc94949927a2e3d21dde202 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:30.810823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:30.814581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:30.848619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e774561-cd81-4f60-89af-7917f833a785 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:30.861937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:30.909988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:30.912078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:30.947174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:30.948118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:30.948392 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:30.948664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] PATH_INFO: `/v3/projects/fff3e5aedee14e2d8ff92a28dae5b963` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:30.949988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff3e5aedee14e2d8ff92a28dae5b963)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:30.952120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:30.980122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dc338-fab6-468c-a8d6-65e99cfcd729 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:30.989038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562a8dc-166c-46d4-9e2d-cdfde9feb648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:30.990157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a8dc-166c-46d4-9e2d-cdfde9feb648 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:30.990611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a8dc-166c-46d4-9e2d-cdfde9feb648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:30.990880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a8dc-166c-46d4-9e2d-cdfde9feb648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:30.994586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d562a8dc-166c-46d4-9e2d-cdfde9feb648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:31.008819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562a8dc-166c-46d4-9e2d-cdfde9feb648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.009629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562a8dc-166c-46d4-9e2d-cdfde9feb648 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.022266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1132/2277] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:31.030511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff3e5aedee14e2d8ff92a28dae5b963', 'request_id': 'req-28df93e9-99ab-422d-a4dd-9d13f5656f99'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:31.039890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff3e5aedee14e2d8ff92a28dae5b963', 'request_id': 'req-28df93e9-99ab-422d-a4dd-9d13f5656f99'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:31.048141 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff3e5aedee14e2d8ff92a28dae5b963', 'request_id': 'req-28df93e9-99ab-422d-a4dd-9d13f5656f99'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:31.064093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] The token cache is being invalidate because project fff3e5aedee14e2d8ff92a28dae5b963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:31.064390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:31.065013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-28df93e9-99ab-422d-a4dd-9d13f5656f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:31.066609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1146/2278] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:30 2022] DELETE /identity/v3/projects/fff3e5aedee14e2d8ff92a28dae5b963 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:31.071683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e016c74-7de4-4409-9dc6-a0210148579f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.071992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e016c74-7de4-4409-9dc6-a0210148579f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.072268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e016c74-7de4-4409-9dc6-a0210148579f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.091430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2e016c74-7de4-4409-9dc6-a0210148579f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:31.092640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2e016c74-7de4-4409-9dc6-a0210148579f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:31.149827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:31.185316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1133/2279] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:31.193113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b295dc-eba8-4450-94be-8ce68c440bd4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.193841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b295dc-eba8-4450-94be-8ce68c440bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.194353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b295dc-eba8-4450-94be-8ce68c440bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.215463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-98b295dc-eba8-4450-94be-8ce68c440bd4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:31.216706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-98b295dc-eba8-4450-94be-8ce68c440bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:31.227317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1147/2280] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:31.231978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e016c74-7de4-4409-9dc6-a0210148579f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.241939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e016c74-7de4-4409-9dc6-a0210148579f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:31.246616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:31.247537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.247939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.248225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.249353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:31.249667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.252005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.266353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1134/2281] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:19:31.270741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b295dc-eba8-4450-94be-8ce68c440bd4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.281414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b295dc-eba8-4450-94be-8ce68c440bd4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:31.285810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:31.286569 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.286913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.287252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.288023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.290050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.324014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1148/2282] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:31 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:31.325202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.341411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb128e7-1ecf-4af0-afd5-a984cf930728 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:31.412884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:31.413742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.414089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.414434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.415434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.417632 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.450431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1135/2283] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:31 2022] POST /identity/v3/users => generated 282 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:31.454639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.466693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b4cf6-7f8e-40aa-bbb8-35072d774641 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:31.476104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:31.476340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.476663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.477009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.477841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.479544 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.501612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1149/2284] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:31.501838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.513188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d3145-56d8-45e7-8673-9513e9305e6e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:31.531447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a649f-7f02-4322-b82a-d7d74c62de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:31.532969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a649f-7f02-4322-b82a-d7d74c62de14 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.533291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a649f-7f02-4322-b82a-d7d74c62de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.533576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a649f-7f02-4322-b82a-d7d74c62de14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.534345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a649f-7f02-4322-b82a-d7d74c62de14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.536011 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a649f-7f02-4322-b82a-d7d74c62de14 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.562446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1136/2285] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:31.566886 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.581228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492aced-6596-4780-9471-512f65871cc5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:31.589760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:31.590184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.590464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.590742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] PATH_INFO: `/v3/projects/27b660ad3e1749a4b53fa5d770f74c12/users/cdc739aa0fa64561967394eebcb5f12f/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.613096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b660ad3e1749a4b53fa5d770f74c12, user_id=cdc739aa0fa64561967394eebcb5f12f, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.615790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.642338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1150/2286] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:31 2022] PUT /identity/v3/projects/27b660ad3e1749a4b53fa5d770f74c12/users/cdc739aa0fa64561967394eebcb5f12f/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:31.645465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dfa6c-d54a-4c31-9fb1-d00a9a792704 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.645785 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dfa6c-d54a-4c31-9fb1-d00a9a792704 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.646069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dfa6c-d54a-4c31-9fb1-d00a9a792704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.692831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-413dfa6c-d54a-4c31-9fb1-d00a9a792704 None None] MFA Rules not processed for user `cdc739aa0fa64561967394eebcb5f12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:31.694110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-413dfa6c-d54a-4c31-9fb1-d00a9a792704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:31.796302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1137/2287] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:31.800640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.849540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:31.882827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03369b-d4f3-4cf3-850b-d773b5a7c85b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:31.889214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:31.890006 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:31.890346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:31.890852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:31.891653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:31.893938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:31.923495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1151/2288] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:31 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:31.927166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dfa6c-d54a-4c31-9fb1-d00a9a792704 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:31.938813 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dfa6c-d54a-4c31-9fb1-d00a9a792704 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:32.012992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:32.053315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:32.054515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:32.055019 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:32.055518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:32.056449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:32.058875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:32.087450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1138/2289] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:31 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:32.091477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:32.105983 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94572d78-baa7-47e7-a088-c773904e2a8f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:32.111874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:32.112105 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:32.112210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:32.112473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:32.113228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:32.115013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:32.128823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1152/2290] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:32.133736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:32.153032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36828f3a-26aa-432b-832f-3a842c9a96c3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:32.166076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:32.167123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:32.168325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:32.168765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] PATH_INFO: `/v3/projects/9ca020c3ea0d4ab895bae1a946daf313/users/aac4648c784f4bf8b9cf25cde2d8c0f5/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:32.210103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca020c3ea0d4ab895bae1a946daf313, user_id=aac4648c784f4bf8b9cf25cde2d8c0f5, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:32.222797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:32.243998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1139/2291] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:32 2022] PUT /identity/v3/projects/9ca020c3ea0d4ab895bae1a946daf313/users/aac4648c784f4bf8b9cf25cde2d8c0f5/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:32.251148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:32.262892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17949d-4ab7-4e3c-8485-cccf9d567eff admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:32.392399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1eae9-7f5e-4fd3-9665-e42420261f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:32.393298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1eae9-7f5e-4fd3-9665-e42420261f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:32.393689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1eae9-7f5e-4fd3-9665-e42420261f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:32.394064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1eae9-7f5e-4fd3-9665-e42420261f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:32.394906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1eae9-7f5e-4fd3-9665-e42420261f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:32.396644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1eae9-7f5e-4fd3-9665-e42420261f90 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:32.413295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1153/2292] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:32.417553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:32.430459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3501c-fe5c-414c-a098-03fd933780c8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:32.435594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:32.436749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:32.438677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:32.439255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] PATH_INFO: `/v3/projects/9ca020c3ea0d4ab895bae1a946daf313/users/aac4648c784f4bf8b9cf25cde2d8c0f5/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:32.442094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca020c3ea0d4ab895bae1a946daf313, user_id=aac4648c784f4bf8b9cf25cde2d8c0f5, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:32.445378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:32.477330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1140/2293] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:32 2022] PUT /identity/v3/projects/9ca020c3ea0d4ab895bae1a946daf313/users/aac4648c784f4bf8b9cf25cde2d8c0f5/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:32.479268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4658-f0f7-4849-aa00-78ff1827aefa None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:32.479661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4658-f0f7-4849-aa00-78ff1827aefa None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:32.480038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4658-f0f7-4849-aa00-78ff1827aefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:32.513513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-873d4658-f0f7-4849-aa00-78ff1827aefa None None] MFA Rules not processed for user `aac4648c784f4bf8b9cf25cde2d8c0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:32.514793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-873d4658-f0f7-4849-aa00-78ff1827aefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:32.615981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1154/2294] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:32.629308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:32.630617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:32.717909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913d4db-6e09-4075-bbb2-5887d7333124 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:32.721817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e810a7-3a33-4860-94ad-01630bdd3efd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:32.722552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e810a7-3a33-4860-94ad-01630bdd3efd service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:32.722839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e810a7-3a33-4860-94ad-01630bdd3efd service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:32.723114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e810a7-3a33-4860-94ad-01630bdd3efd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:32.732954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e810a7-3a33-4860-94ad-01630bdd3efd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:32.734597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e810a7-3a33-4860-94ad-01630bdd3efd service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:32.757907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1141/2295] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:33.401615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4658-f0f7-4849-aa00-78ff1827aefa None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:33.472828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:33.519006 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4658-f0f7-4849-aa00-78ff1827aefa None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:33.527401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:33.528326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:33.528611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:33.528890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] PATH_INFO: `/v3/users/cdc739aa0fa64561967394eebcb5f12f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:33.530386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc739aa0fa64561967394eebcb5f12f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:33.533302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:33.616386 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc739aa0fa64561967394eebcb5f12f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:33.617251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc739aa0fa64561967394eebcb5f12f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:33.617861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc739aa0fa64561967394eebcb5f12f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:33.636776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc739aa0fa64561967394eebcb5f12f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:33.661916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc739aa0fa64561967394eebcb5f12f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:33.693396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1155/2296] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:33 2022] DELETE /identity/v3/users/cdc739aa0fa64561967394eebcb5f12f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:33.699743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:33.700359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:33.700701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:33.758884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:33.758884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:33.876702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1142/2297] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:33.886977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:33.888412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:34.016434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a01b70-0cb7-46a5-b45d-79cb8b9872a4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:34.020432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:34.021265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:34.021849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:34.021986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:34.034012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:34.036519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:34.051212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1156/2298] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:34.642740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:34.644262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:34.671023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ce3aa-4ea1-4883-bad1-6896b8c2727f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:34.677609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:34.679691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:34.680080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:34.680458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] PATH_INFO: `/v3/projects/27b660ad3e1749a4b53fa5d770f74c12` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:34.681879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b660ad3e1749a4b53fa5d770f74c12)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:34.684040 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:34.722933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:34.747927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b660ad3e1749a4b53fa5d770f74c12', 'request_id': 'req-f42d9836-1062-493b-b3f7-17f7f97e41b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.756392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b660ad3e1749a4b53fa5d770f74c12', 'request_id': 'req-f42d9836-1062-493b-b3f7-17f7f97e41b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.763378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b660ad3e1749a4b53fa5d770f74c12', 'request_id': 'req-f42d9836-1062-493b-b3f7-17f7f97e41b3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.774993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] The token cache is being invalidate because project 27b660ad3e1749a4b53fa5d770f74c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:34.775238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.776002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.777614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1143/2299] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:34 2022] DELETE /identity/v3/projects/27b660ad3e1749a4b53fa5d770f74c12 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:34.782358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:34.792829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb9bf6-e9fc-4ec9-96c2-35bef5d5a068 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:34.864772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:34.871972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:34.872935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:34.873428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:34.873890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] PATH_INFO: `/v3/users/aac4648c784f4bf8b9cf25cde2d8c0f5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:34.876040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac4648c784f4bf8b9cf25cde2d8c0f5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:34.878646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:34.929628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac4648c784f4bf8b9cf25cde2d8c0f5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.930327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac4648c784f4bf8b9cf25cde2d8c0f5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.930980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac4648c784f4bf8b9cf25cde2d8c0f5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.946362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac4648c784f4bf8b9cf25cde2d8c0f5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.955364 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fef9f43d-5bea-490f-bf7f-b7714f11c531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac4648c784f4bf8b9cf25cde2d8c0f5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:34.972857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1157/2300] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:34 2022] DELETE /identity/v3/users/aac4648c784f4bf8b9cf25cde2d8c0f5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:35.469826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:35.471802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:35.493904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d9836-1062-493b-b3f7-17f7f97e41b3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:35.590635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:35.591633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:35.591926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:35.592200 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] PATH_INFO: `/v3/projects/9ca020c3ea0d4ab895bae1a946daf313` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:35.594763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca020c3ea0d4ab895bae1a946daf313)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:35.620528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:35.691330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca020c3ea0d4ab895bae1a946daf313', 'request_id': 'req-eb6153fc-a605-4fe7-b2fc-0ce142d93706'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.699358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca020c3ea0d4ab895bae1a946daf313', 'request_id': 'req-eb6153fc-a605-4fe7-b2fc-0ce142d93706'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.707489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca020c3ea0d4ab895bae1a946daf313', 'request_id': 'req-eb6153fc-a605-4fe7-b2fc-0ce142d93706'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.722497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] The token cache is being invalidate because project 9ca020c3ea0d4ab895bae1a946daf313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:35.722760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.723642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.725269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1144/2301] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:35 2022] DELETE /identity/v3/projects/9ca020c3ea0d4ab895bae1a946daf313 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:35.730310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b437b-8232-4ae7-910e-54674a7afc44 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:35.730630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b437b-8232-4ae7-910e-54674a7afc44 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:35.730913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b437b-8232-4ae7-910e-54674a7afc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:35.749542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cc0b437b-8232-4ae7-910e-54674a7afc44 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:35.750876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cc0b437b-8232-4ae7-910e-54674a7afc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:35.777022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:35.778782 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:35.829625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1158/2302] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:35.831642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:35.837647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001e0c3-8e76-419d-a409-c693329ee0fd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:35.837943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001e0c3-8e76-419d-a409-c693329ee0fd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:35.838221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001e0c3-8e76-419d-a409-c693329ee0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:35.854905 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6153fc-a605-4fe7-b2fc-0ce142d93706 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:35.859733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8001e0c3-8e76-419d-a409-c693329ee0fd None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:35.860142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:35.861040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8001e0c3-8e76-419d-a409-c693329ee0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:35.861214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:35.861479 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:35.861765 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] PATH_INFO: `/v3/users/583143af14a645238262214515ee148f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:35.862993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=583143af14a645238262214515ee148f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:35.864945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:35.869835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1159/2303] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:35.873865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001e0c3-8e76-419d-a409-c693329ee0fd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:35.884177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001e0c3-8e76-419d-a409-c693329ee0fd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:35.889539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:35.890198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:35.890540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:35.890809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:35.891530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:35.891843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:35.893596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:35.906580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1160/2304] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:19:35.910848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:35.919323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583143af14a645238262214515ee148f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.919992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583143af14a645238262214515ee148f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.921271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f431d-fadd-4e9e-bdc3-50e5f45587e2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:35.926496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:35.927171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:35.927451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:35.927720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:35.928413 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:35.930337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:35.932823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583143af14a645238262214515ee148f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.933663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583143af14a645238262214515ee148f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.940689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583143af14a645238262214515ee148f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:35.955426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1145/2305] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:35 2022] DELETE /identity/v3/users/583143af14a645238262214515ee148f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:35.959033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1161/2306] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:35 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:35.962876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:35.964413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.040524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92f1b2-424b-4b39-9b5b-ac8639bc21f0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.101879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.102564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.102842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.103115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.103817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.105787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.121475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:36.129790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1146/2307] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:35 2022] POST /identity/v3/users => generated 282 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:36.133620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.145198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac50c89-8e44-4b97-ad56-a880ae2c64cb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.151604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.152318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.152612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.152887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.153657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.155269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.159737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.161656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.167987 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1162/2308] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:36.171727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.180594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113fb53-aa9b-4355-9c51-5ebaf77ddb8c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.185468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.186331 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b6a7c-09dc-49a3-9ec5-99a9798057c9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.186495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.186602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.186860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] PATH_INFO: `/v3/projects/6be635735d174893be20f99110cacbfa` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.188888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be635735d174893be20f99110cacbfa)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.191859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.192665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.193425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.193774 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.194093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.194781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.196383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.207154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1163/2309] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:36.211283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.221286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56287303-5130-4fd4-89b4-9e821e2bf11c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.226451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b563d-d1ec-4778-ad9c-f4024a9fa4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.227349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b563d-d1ec-4778-ad9c-f4024a9fa4de admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.227631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b563d-d1ec-4778-ad9c-f4024a9fa4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.228034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b563d-d1ec-4778-ad9c-f4024a9fa4de admin admin] PATH_INFO: `/v3/projects/63fe07e5855349b8aff8d8a68c37e649/users/2195beda0ccd46a7acd8652f39d98927/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.239652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:36.244049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be635735d174893be20f99110cacbfa', 'request_id': 'req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:36.244388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b563d-d1ec-4778-ad9c-f4024a9fa4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fe07e5855349b8aff8d8a68c37e649, user_id=2195beda0ccd46a7acd8652f39d98927, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.247285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b563d-d1ec-4778-ad9c-f4024a9fa4de admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.252464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be635735d174893be20f99110cacbfa', 'request_id': 'req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:36.260663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be635735d174893be20f99110cacbfa', 'request_id': 'req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:36.263100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1164/2310] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:36 2022] PUT /identity/v3/projects/63fe07e5855349b8aff8d8a68c37e649/users/2195beda0ccd46a7acd8652f39d98927/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:36.273407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.273696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.273969 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.274777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] The token cache is being invalidate because project 6be635735d174893be20f99110cacbfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:36.275068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:36.275806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-23c89241-3cd0-45aa-a000-58ee5dc3d1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:36.277550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1147/2311] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:36 2022] DELETE /identity/v3/projects/6be635735d174893be20f99110cacbfa => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:36.283887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e186524-bb5a-44c2-8d84-5eba4737fbd2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.284330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e186524-bb5a-44c2-8d84-5eba4737fbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.284776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e186524-bb5a-44c2-8d84-5eba4737fbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.307212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2e186524-bb5a-44c2-8d84-5eba4737fbd2 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:36.308408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2e186524-bb5a-44c2-8d84-5eba4737fbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.310121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] MFA Rules not processed for user `2195beda0ccd46a7acd8652f39d98927`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:36.311270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.371892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1148/2312] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:36.383038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1165/2313] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:36.383222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fcd5-0176-45db-8455-6fdcc6b65d05 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.383222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fcd5-0176-45db-8455-6fdcc6b65d05 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.383222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fcd5-0176-45db-8455-6fdcc6b65d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.384064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.385439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.398842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb5050-c9d9-4899-ad70-887041635761 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.401101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ac25fcd5-0176-45db-8455-6fdcc6b65d05 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:36.402642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ac25fcd5-0176-45db-8455-6fdcc6b65d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.403489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.404149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.404438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.404716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.405515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.407847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.411159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1149/2314] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:36.415307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25fcd5-0176-45db-8455-6fdcc6b65d05 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.424487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25fcd5-0176-45db-8455-6fdcc6b65d05 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.430140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.430857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.431134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.431400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.432124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:36.432423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.434040 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.435374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1166/2315] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:36.439314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.446143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1150/2316] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:19:36.450353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.460266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d744-9785-444b-beb7-004d4353b583 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.502317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39ad62-d7f7-4914-b892-eb08bd975f8b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.507280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.507977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.508257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.508527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.509270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.511230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.517593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.518280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.518556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.518826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.519509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.521426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.526323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:36.536567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1167/2317] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:36.537665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.548554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609ba41-8008-4fcb-acce-352cd4f00116 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.552023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1151/2318] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:36.554057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.554734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.554984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.555271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.556002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.556245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.557673 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.571352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1168/2319] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:36.572382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.582430 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec3095-9e8c-4934-9bd3-e556585d198b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.614694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:36.620832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30101962-46be-4741-b3df-a9b5f78df0d1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.626475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.627302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.627692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.627972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.628664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.630738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.640470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.641530 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.641811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.642087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] PATH_INFO: `/v3/projects/eff6d755352648559e5fde0abc35e46e/users/00dd854ed9e940cba222b953da5a4167/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.653839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1152/2320] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:36.658561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.660522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff6d755352648559e5fde0abc35e46e, user_id=00dd854ed9e940cba222b953da5a4167, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.663218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.668257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc377cf-5083-4e24-9848-0e179a2263df admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.673126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.673789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.674069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.674341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.675070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.676683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.678242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1169/2321] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:36 2022] PUT /identity/v3/projects/eff6d755352648559e5fde0abc35e46e/users/00dd854ed9e940cba222b953da5a4167/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:36.681932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.691099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1153/2322] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:36.694955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.705365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb62d8b-a769-424d-9dd0-6e128f671a7c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.748391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f69cc-b0b3-4e3f-93a1-46458c2aeab7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.754838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.755460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.755741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.756020 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.756759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.758417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.769511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1170/2323] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:36.773485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.775616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.777134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.777472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.777752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.778503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.780115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.784766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fecec8-e683-4f1f-ad61-3667d702fa8b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.789498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6135bd-f689-4f5b-90ad-1aa0a3e79fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.790654 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6135bd-f689-4f5b-90ad-1aa0a3e79fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.791019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6135bd-f689-4f5b-90ad-1aa0a3e79fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.791161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1154/2324] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:36.791407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6135bd-f689-4f5b-90ad-1aa0a3e79fa0 admin admin] PATH_INFO: `/v3/projects/eff6d755352648559e5fde0abc35e46e/users/00dd854ed9e940cba222b953da5a4167/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.794054 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6135bd-f689-4f5b-90ad-1aa0a3e79fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff6d755352648559e5fde0abc35e46e, user_id=00dd854ed9e940cba222b953da5a4167, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.795302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.796976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6135bd-f689-4f5b-90ad-1aa0a3e79fa0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.805284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb53e-2767-4eaa-b9c5-7051e8011476 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:36.808747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1171/2325] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:36 2022] PUT /identity/v3/projects/eff6d755352648559e5fde0abc35e46e/users/00dd854ed9e940cba222b953da5a4167/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:36.820495 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.821238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.821874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.854016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] MFA Rules not processed for user `00dd854ed9e940cba222b953da5a4167`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:36.855310 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.876025 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7255b52-5332-4afa-97fb-38a387b6037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:36.877009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7255b52-5332-4afa-97fb-38a387b6037a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.877727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7255b52-5332-4afa-97fb-38a387b6037a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.878063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7255b52-5332-4afa-97fb-38a387b6037a admin admin] PATH_INFO: `/v3/projects/f63e110fcf424af19df384d63513cd94/users/9333cc09223b42919cb067ed39e81f3f/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.881384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:36.894737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7255b52-5332-4afa-97fb-38a387b6037a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63e110fcf424af19df384d63513cd94, user_id=9333cc09223b42919cb067ed39e81f3f, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:36.897468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7255b52-5332-4afa-97fb-38a387b6037a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:36.911598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1155/2326] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:36 2022] PUT /identity/v3/projects/f63e110fcf424af19df384d63513cd94/users/9333cc09223b42919cb067ed39e81f3f/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:36.922066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1681bc-dc64-4b2e-9ee5-a106e2f4a7a3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:36.922387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1681bc-dc64-4b2e-9ee5-a106e2f4a7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:36.922658 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1681bc-dc64-4b2e-9ee5-a106e2f4a7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:36.923256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1172/2327] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:36.934780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:36.936196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.950374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8a1681bc-dc64-4b2e-9ee5-a106e2f4a7a3 None None] MFA Rules not processed for user `9333cc09223b42919cb067ed39e81f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:36.951620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8a1681bc-dc64-4b2e-9ee5-a106e2f4a7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:36.970104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:36.998319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e55f-66b4-4657-8b23-da8490e6b9e6 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.016285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1156/2328] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:37.020598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1681bc-dc64-4b2e-9ee5-a106e2f4a7a3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.051196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.051847 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.052124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.052399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.082083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1681bc-dc64-4b2e-9ee5-a106e2f4a7a3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.086829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.087494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.087770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.088043 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.088731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.090674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.112783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.114314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.117589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1157/2329] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:37 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:37.121604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.128351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1173/2330] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:19:37.199121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe54ba-edd3-41eb-8af9-c62410b6f971 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.203392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.204216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.204791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.205428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.206918 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.207431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.208153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.216264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:37.217387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40556e1-8897-428b-8cad-ab6e0d251c5c service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.233064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1158/2331] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:37 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:37.236968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.251142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9816931-c6a2-4e60-95f8-1bf4a812c0b7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.256676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.257388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.257762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.258076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.258829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.260461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.272053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1159/2332] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:37.276401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.280387 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.281844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.282259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.282532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.289514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2240aad-f880-4b0b-b45a-30bf48c6afc8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.295158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.296440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.296744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.297022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] PATH_INFO: `/v3/projects/bbc46451ba244f2c9d64d19ec531bcf9/users/de32986ec2e042bea0b6656c241860c0/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.313209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc46451ba244f2c9d64d19ec531bcf9, user_id=de32986ec2e042bea0b6656c241860c0, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.315877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.329536 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:37.330309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1160/2333] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:37 2022] PUT /identity/v3/projects/bbc46451ba244f2c9d64d19ec531bcf9/users/de32986ec2e042bea0b6656c241860c0/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:37.334444 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.342809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.344351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.353764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1174/2334] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:37 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:19:37.396711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729643b-9214-4feb-a5a1-4a80c5c16102 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.401739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a5b3d-a5b4-4993-adf8-a96d6a32d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.402378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a5b3d-a5b4-4993-adf8-a96d6a32d0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.402689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a5b3d-a5b4-4993-adf8-a96d6a32d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.402936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a5b3d-a5b4-4993-adf8-a96d6a32d0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.403661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a5b3d-a5b4-4993-adf8-a96d6a32d0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.405292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a5b3d-a5b4-4993-adf8-a96d6a32d0b7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.415994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1161/2335] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:37.420739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.432170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81ffdd-a870-407c-b619-c20b82909b36 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.437570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.438507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.438784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.439057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] PATH_INFO: `/v3/projects/bbc46451ba244f2c9d64d19ec531bcf9/users/de32986ec2e042bea0b6656c241860c0/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.441267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc46451ba244f2c9d64d19ec531bcf9, user_id=de32986ec2e042bea0b6656c241860c0, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.444053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.455766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1175/2336] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:37 2022] PUT /identity/v3/projects/bbc46451ba244f2c9d64d19ec531bcf9/users/de32986ec2e042bea0b6656c241860c0/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:37.466387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ac881-ee33-4ab0-bcdb-0f9605a62f85 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.466681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ac881-ee33-4ab0-bcdb-0f9605a62f85 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.466962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ac881-ee33-4ab0-bcdb-0f9605a62f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.498572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f93ac881-ee33-4ab0-bcdb-0f9605a62f85 None None] MFA Rules not processed for user `de32986ec2e042bea0b6656c241860c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:37.499895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f93ac881-ee33-4ab0-bcdb-0f9605a62f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:37.568669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1162/2337] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:37.582066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.647136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0570957-7dae-4900-b137-a5a81dd5cf82 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.651466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916558f-a28d-4143-8572-b375a16d8b77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.652120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916558f-a28d-4143-8572-b375a16d8b77 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.652402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916558f-a28d-4143-8572-b375a16d8b77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.652677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916558f-a28d-4143-8572-b375a16d8b77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.663011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916558f-a28d-4143-8572-b375a16d8b77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.664459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916558f-a28d-4143-8572-b375a16d8b77 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.674207 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1176/2338] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:37.712586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ac881-ee33-4ab0-bcdb-0f9605a62f85 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.755467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:37.783219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ac881-ee33-4ab0-bcdb-0f9605a62f85 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:37.788093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:37.788995 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.789302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.789575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] PATH_INFO: `/v3/users/9333cc09223b42919cb067ed39e81f3f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.790811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9333cc09223b42919cb067ed39e81f3f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:37.792751 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:37.844590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9333cc09223b42919cb067ed39e81f3f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:37.845208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9333cc09223b42919cb067ed39e81f3f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:37.857836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9333cc09223b42919cb067ed39e81f3f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:37.858553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9333cc09223b42919cb067ed39e81f3f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:37.866431 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9333cc09223b42919cb067ed39e81f3f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:37.880552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1163/2339] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:37 2022] DELETE /identity/v3/users/9333cc09223b42919cb067ed39e81f3f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:37.885295 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:37.885777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:37.886072 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:37.904525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:37.906751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:37.967948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1177/2340] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:37.978583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:37.980165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:38.041109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e7dbc-5b8a-469b-9660-1a2aea9dc80d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:38.044772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:38.045449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:38.045728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:38.045998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:38.055087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:38.056505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:38.066026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1164/2341] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:38.343370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:38.345037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:38.357903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21d653-09ed-4bd9-949b-b85bc7af9c0a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:38.364068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:38.364970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:38.365272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:38.365549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] PATH_INFO: `/v3/projects/f63e110fcf424af19df384d63513cd94` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:38.366872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63e110fcf424af19df384d63513cd94)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:38.368911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:38.419527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63e110fcf424af19df384d63513cd94', 'request_id': 'req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.425891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63e110fcf424af19df384d63513cd94', 'request_id': 'req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.431324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63e110fcf424af19df384d63513cd94', 'request_id': 'req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.442049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:38.446627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] The token cache is being invalidate because project f63e110fcf424af19df384d63513cd94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:38.446925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.447829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.449923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1178/2342] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:38 2022] DELETE /identity/v3/projects/f63e110fcf424af19df384d63513cd94 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:38.454425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:38.464465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344af5f-9d78-4828-91fb-18b586379767 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:38.519644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:38.527285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:38.528611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:38.529207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:38.530061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] PATH_INFO: `/v3/users/de32986ec2e042bea0b6656c241860c0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:38.531688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de32986ec2e042bea0b6656c241860c0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:38.533760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:38.584496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de32986ec2e042bea0b6656c241860c0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.585174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de32986ec2e042bea0b6656c241860c0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.601347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de32986ec2e042bea0b6656c241860c0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.602157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de32986ec2e042bea0b6656c241860c0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.610575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9af21be3-b688-4871-834e-33b7e5410362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de32986ec2e042bea0b6656c241860c0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:38.628029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1165/2343] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:38 2022] DELETE /identity/v3/users/de32986ec2e042bea0b6656c241860c0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:38.931237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:38.932677 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:38.946339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292e465-5a2d-4b8d-a4bb-2b3f54f8060f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.003923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.004839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.005166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.005430 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] PATH_INFO: `/v3/projects/bbc46451ba244f2c9d64d19ec531bcf9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.006620 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc46451ba244f2c9d64d19ec531bcf9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.008556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.065389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc46451ba244f2c9d64d19ec531bcf9', 'request_id': 'req-f7a38c3b-9233-4679-b7aa-2b6239789656'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:39.073272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc46451ba244f2c9d64d19ec531bcf9', 'request_id': 'req-f7a38c3b-9233-4679-b7aa-2b6239789656'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:39.079839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc46451ba244f2c9d64d19ec531bcf9', 'request_id': 'req-f7a38c3b-9233-4679-b7aa-2b6239789656'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:39.095876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] The token cache is being invalidate because project bbc46451ba244f2c9d64d19ec531bcf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:39.096308 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:39.097018 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f7a38c3b-9233-4679-b7aa-2b6239789656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:39.098905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1179/2344] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:38 2022] DELETE /identity/v3/projects/bbc46451ba244f2c9d64d19ec531bcf9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:39.103713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97873-a38e-43e4-ad76-3b1c7ac81645 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.104254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97873-a38e-43e4-ad76-3b1c7ac81645 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.104254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97873-a38e-43e4-ad76-3b1c7ac81645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.124288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2ac97873-a38e-43e4-ad76-3b1c7ac81645 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:39.125738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2ac97873-a38e-43e4-ad76-3b1c7ac81645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:39.193770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1166/2345] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:39.201512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7d8ab-71ab-4c66-98fc-990e1283d94a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.201854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7d8ab-71ab-4c66-98fc-990e1283d94a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.202139 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7d8ab-71ab-4c66-98fc-990e1283d94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.224097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-16f7d8ab-71ab-4c66-98fc-990e1283d94a None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:39.225363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-16f7d8ab-71ab-4c66-98fc-990e1283d94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:39.233564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1180/2346] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:39.239047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac97873-a38e-43e4-ad76-3b1c7ac81645 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.250924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac97873-a38e-43e4-ad76-3b1c7ac81645 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.256140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.256819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.257168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.257469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.258247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:39.258570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.260405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.273264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1167/2347] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:19:39.278155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7d8ab-71ab-4c66-98fc-990e1283d94a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.288016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7d8ab-71ab-4c66-98fc-990e1283d94a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.293977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.294678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.295506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.295506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.295905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.298007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.325539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1181/2348] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:39 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:39.330015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.340063 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e6f27-11d5-498c-8be8-d50aee50b5d9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.401376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.402343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.402712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.403060 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.404028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.407325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.420302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:39.438014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1168/2349] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:39 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:39.442153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.453171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68a5a6-c2fa-4a64-92b6-fdce55db8146 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.458805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.459414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.459694 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.459968 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.460695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.462338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.473452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1182/2350] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:39.477716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.488141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81d50-c490-4c30-bffc-d176074a336a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.493408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b34f32-5d3c-4f23-a6fd-1409d03c123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.494044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34f32-5d3c-4f23-a6fd-1409d03c123d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.494325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34f32-5d3c-4f23-a6fd-1409d03c123d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.494608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34f32-5d3c-4f23-a6fd-1409d03c123d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.495336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b34f32-5d3c-4f23-a6fd-1409d03c123d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.496941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b34f32-5d3c-4f23-a6fd-1409d03c123d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.507605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1169/2351] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:39.511779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.522296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73acd7-53e5-4feb-8779-9a439bb80324 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.527970 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.528889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.529200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.529476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] PATH_INFO: `/v3/projects/33fd752645e04c5f99f7fc86df083062/users/e58f7d421c2946c2b0c9c48a7c72cd24/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.546409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fd752645e04c5f99f7fc86df083062, user_id=e58f7d421c2946c2b0c9c48a7c72cd24, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.549079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.564218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1183/2352] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:39 2022] PUT /identity/v3/projects/33fd752645e04c5f99f7fc86df083062/users/e58f7d421c2946c2b0c9c48a7c72cd24/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:39.574831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cdc78-be45-4d72-95d4-48f5bede6454 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.575120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cdc78-be45-4d72-95d4-48f5bede6454 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.575396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cdc78-be45-4d72-95d4-48f5bede6454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.602542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5d2cdc78-be45-4d72-95d4-48f5bede6454 None None] MFA Rules not processed for user `e58f7d421c2946c2b0c9c48a7c72cd24`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:39.603759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5d2cdc78-be45-4d72-95d4-48f5bede6454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:39.669017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1170/2353] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:39.673498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.704830 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:39.737657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbac4ad-b164-4146-a8d3-05060f029646 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.742238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.742900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.743178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.743448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.744167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.746097 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.774972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1184/2354] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:39 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:39.779472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cdc78-be45-4d72-95d4-48f5bede6454 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.791727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cdc78-be45-4d72-95d4-48f5bede6454 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.853855 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.854543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.854858 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.855111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.855829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.857801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.880382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1171/2355] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:39 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:39.884623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.895972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eecb12-e4e5-4684-8467-e18ed320d3d2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.902296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.903007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.903287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.903555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.904594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.906255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.917531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1185/2356] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:39.921957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.932363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde0ffe-ce87-4c91-b1d7-a2680e779019 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:39.937392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:39.938323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:39.938593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:39.938863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] PATH_INFO: `/v3/projects/c840445f174840228aed29874839945e/users/65ab0f5a49c54fa5a14b21a0ba45f137/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:39.957074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c840445f174840228aed29874839945e, user_id=65ab0f5a49c54fa5a14b21a0ba45f137, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:39.960085 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:39.978354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1172/2357] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:39 2022] PUT /identity/v3/projects/c840445f174840228aed29874839945e/users/65ab0f5a49c54fa5a14b21a0ba45f137/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:39.982645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:39.994408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bad67-f2a0-4a90-aa3d-2ae8fe9ea23f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:40.056193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90bde6-18cc-4dae-8494-c48335f7b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:40.056879 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90bde6-18cc-4dae-8494-c48335f7b9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.057195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90bde6-18cc-4dae-8494-c48335f7b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.057476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90bde6-18cc-4dae-8494-c48335f7b9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.058206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90bde6-18cc-4dae-8494-c48335f7b9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:40.059812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90bde6-18cc-4dae-8494-c48335f7b9ef admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:40.071010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1186/2358] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:40.075327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:40.086187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe5ed-d87a-4857-be1a-5bc8bc895d63 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:40.097678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:40.098623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.098900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.099169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] PATH_INFO: `/v3/projects/c840445f174840228aed29874839945e/users/65ab0f5a49c54fa5a14b21a0ba45f137/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.101350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c840445f174840228aed29874839945e, user_id=65ab0f5a49c54fa5a14b21a0ba45f137, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:40.104033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:40.115787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1173/2359] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:40 2022] PUT /identity/v3/projects/c840445f174840228aed29874839945e/users/65ab0f5a49c54fa5a14b21a0ba45f137/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:40.128053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38900877-bad8-41c2-a17e-5062582716dd None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.128342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38900877-bad8-41c2-a17e-5062582716dd None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.128645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38900877-bad8-41c2-a17e-5062582716dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.156675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-38900877-bad8-41c2-a17e-5062582716dd None None] MFA Rules not processed for user `65ab0f5a49c54fa5a14b21a0ba45f137`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:40.157974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-38900877-bad8-41c2-a17e-5062582716dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:40.227935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1187/2360] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:40.242333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:40.244154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:40.282382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:40.326209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e81f66-ee84-4552-9e61-454efd752fe9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:40.329925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:40.330586 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.330877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.331181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.341615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:40.345228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:40.357387 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1174/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:19:40.684648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38900877-bad8-41c2-a17e-5062582716dd None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:40.698960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38900877-bad8-41c2-a17e-5062582716dd None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:40.705971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65ab0f5a49c54fa5a14b21a0ba45f137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c840445f174840228aed29874839945e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:40.706669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.706951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.707230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.707964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:40.708271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:40.709953 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:40.722630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1188/2362] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:19:40.727048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:40.737081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41c934-094d-49d5-968b-653b41c05984 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:40.742903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65ab0f5a49c54fa5a14b21a0ba45f137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c840445f174840228aed29874839945e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:40.743563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.743839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.744110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.744789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:40.746712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:40.774783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1175/2363] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:40 2022] POST /identity/v3/projects => generated 354 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:40.778871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:40.790153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78dd00-b61d-4de4-a750-3e2b555576ab tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:40.795183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:40.849776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65ab0f5a49c54fa5a14b21a0ba45f137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c840445f174840228aed29874839945e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:40.850458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.850751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.851059 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.851813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:40.852159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:40.853828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:40.867133 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1189/2364] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:19:40.907697 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:40.923067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04a467-8403-422b-8c4b-deb0e21c6d27 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:40.931635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65ab0f5a49c54fa5a14b21a0ba45f137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c840445f174840228aed29874839945e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:40.932872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:40.933254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:40.933564 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] PATH_INFO: `/v3/projects/340089c9e6e44666bcdcc39e7260e994` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:40.935529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorizing `identity:delete_project(project_id=340089c9e6e44666bcdcc39e7260e994)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:40.938379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:40.989061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340089c9e6e44666bcdcc39e7260e994', 'request_id': 'req-b4d8b6c9-7879-45c5-a748-fc52c8067e24'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:40.998767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340089c9e6e44666bcdcc39e7260e994', 'request_id': 'req-b4d8b6c9-7879-45c5-a748-fc52c8067e24'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.007730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340089c9e6e44666bcdcc39e7260e994', 'request_id': 'req-b4d8b6c9-7879-45c5-a748-fc52c8067e24'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.020265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] The token cache is being invalidate because project 340089c9e6e44666bcdcc39e7260e994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:41.020525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.021538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b4d8b6c9-7879-45c5-a748-fc52c8067e24 tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.023501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1176/2365] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:40 2022] DELETE /identity/v3/projects/340089c9e6e44666bcdcc39e7260e994 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:41.230357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:41.232324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:41.302506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924350-9ad5-423d-94af-e36916b2301f tempest-VolumeQuotaClassesTest-1890051552 tempest-VolumeQuotaClassesTest-1890051552-project-admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:41.362481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:41.363360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:41.363627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:41.363898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] PATH_INFO: `/v3/users/e58f7d421c2946c2b0c9c48a7c72cd24` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:41.365154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58f7d421c2946c2b0c9c48a7c72cd24)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:41.367074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:41.425858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58f7d421c2946c2b0c9c48a7c72cd24'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.426599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58f7d421c2946c2b0c9c48a7c72cd24'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.427224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58f7d421c2946c2b0c9c48a7c72cd24'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.432150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:41.443049 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58f7d421c2946c2b0c9c48a7c72cd24'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.450753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58f7d421c2946c2b0c9c48a7c72cd24'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:41.465343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1190/2366] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:41 2022] DELETE /identity/v3/users/e58f7d421c2946c2b0c9c48a7c72cd24 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:41.470350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:41.470901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:41.470901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:41.489417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:41.490664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:41.553917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1177/2367] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:41.567675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:41.569238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:41.645214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b0bd7-211a-465a-bf78-5b2a3ce525a1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:41.648864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:41.649607 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:41.649892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:41.650302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:41.659990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:41.661595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:41.671212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1191/2368] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:41.950593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:41.952087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:41.964332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd87a1-5123-4d20-a49a-b9ee1ec8253f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:41.969425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:41.970274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:41.970553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:41.970854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] PATH_INFO: `/v3/projects/33fd752645e04c5f99f7fc86df083062` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:41.972073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33fd752645e04c5f99f7fc86df083062)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:41.974039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:42.004065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:42.028638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33fd752645e04c5f99f7fc86df083062', 'request_id': 'req-71a903e4-6a23-4d8d-be71-7bfc76785b25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.038065 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33fd752645e04c5f99f7fc86df083062', 'request_id': 'req-71a903e4-6a23-4d8d-be71-7bfc76785b25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.044745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33fd752645e04c5f99f7fc86df083062', 'request_id': 'req-71a903e4-6a23-4d8d-be71-7bfc76785b25'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.055064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] The token cache is being invalidate because project 33fd752645e04c5f99f7fc86df083062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:42.055368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.055971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.057561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1178/2369] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:41 2022] DELETE /identity/v3/projects/33fd752645e04c5f99f7fc86df083062 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:42.061743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:42.071411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7a221-209c-4f0b-9876-bb3d7b1e943d service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:42.128827 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:42.129876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:42.130158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:42.130433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] PATH_INFO: `/v3/users/65ab0f5a49c54fa5a14b21a0ba45f137` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:42.131745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ab0f5a49c54fa5a14b21a0ba45f137)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:42.133734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:42.180661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ab0f5a49c54fa5a14b21a0ba45f137'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.181312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ab0f5a49c54fa5a14b21a0ba45f137'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.181909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ab0f5a49c54fa5a14b21a0ba45f137'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.194489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ab0f5a49c54fa5a14b21a0ba45f137'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.202964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-327db612-8063-4493-98f2-cd56c6fc76ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ab0f5a49c54fa5a14b21a0ba45f137'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.204699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:42.218600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1192/2370] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:42 2022] DELETE /identity/v3/users/65ab0f5a49c54fa5a14b21a0ba45f137 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:42.504394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:42.506050 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:42.519526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a903e4-6a23-4d8d-be71-7bfc76785b25 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:42.580033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:42.580858 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:42.581172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:42.581449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] PATH_INFO: `/v3/projects/c840445f174840228aed29874839945e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:42.582726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c840445f174840228aed29874839945e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:42.584689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:42.642749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c840445f174840228aed29874839945e', 'request_id': 'req-e5289322-3082-4723-af6c-841bafa3416a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.656969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c840445f174840228aed29874839945e', 'request_id': 'req-e5289322-3082-4723-af6c-841bafa3416a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.670430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c840445f174840228aed29874839945e', 'request_id': 'req-e5289322-3082-4723-af6c-841bafa3416a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.688815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] The token cache is being invalidate because project c840445f174840228aed29874839945e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:42.689583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.690258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e5289322-3082-4723-af6c-841bafa3416a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:42.691869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1179/2371] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:42 2022] DELETE /identity/v3/projects/c840445f174840228aed29874839945e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:42.696962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58115d-cba6-4556-8bdb-065fc156addc None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:42.697464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58115d-cba6-4556-8bdb-065fc156addc None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:42.697562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58115d-cba6-4556-8bdb-065fc156addc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:42.721640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8c58115d-cba6-4556-8bdb-065fc156addc None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:42.722846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8c58115d-cba6-4556-8bdb-065fc156addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:42.784928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1193/2372] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:42.792795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfe533-6fc7-480a-a5cb-6395750c9672 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:42.793107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfe533-6fc7-480a-a5cb-6395750c9672 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:42.793384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfe533-6fc7-480a-a5cb-6395750c9672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:42.814578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1cbfe533-6fc7-480a-a5cb-6395750c9672 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:42.815778 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1cbfe533-6fc7-480a-a5cb-6395750c9672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:42.824732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1180/2373] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:42.829173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58115d-cba6-4556-8bdb-065fc156addc None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:42.839222 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58115d-cba6-4556-8bdb-065fc156addc None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:42.844466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:42.845149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:42.845429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:42.845702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:42.846456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:42.846754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:42.848372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:42.860272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1194/2374] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:19:42.864562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfe533-6fc7-480a-a5cb-6395750c9672 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:42.874210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfe533-6fc7-480a-a5cb-6395750c9672 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:42.879562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:42.880208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:42.880495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:42.880762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:42.881503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:42.883432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:42.911219 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1181/2375] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:42 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:42.915454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:42.926345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a290279-3cac-4f10-93b1-6c53dff86ec3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:42.992679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:42.993401 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:42.993728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:42.994004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:42.994804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:42.997041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.020103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1195/2376] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:42 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:43.024362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.035093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e13c23-10f5-4d16-9f95-a56e5de41a4e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.039727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.040426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.040685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.040958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.041717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.043315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.053830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1182/2377] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:43.057661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.067815 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cd906-193d-4d9a-a887-2db68ed91c7d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.073143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5a7e7-8c8d-4a64-b5c7-ba0343c7d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.073810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a7e7-8c8d-4a64-b5c7-ba0343c7d21e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.074093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a7e7-8c8d-4a64-b5c7-ba0343c7d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.074360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a7e7-8c8d-4a64-b5c7-ba0343c7d21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.075092 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5a7e7-8c8d-4a64-b5c7-ba0343c7d21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.076683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5a7e7-8c8d-4a64-b5c7-ba0343c7d21e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.087093 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1196/2378] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:43.091288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.096639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:43.103089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddf7b3-f778-4670-847b-61120a56fde0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.109014 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.110000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.110259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.110540 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] PATH_INFO: `/v3/projects/59c3aa2f2b864edf9ab8ba847dda239d/users/0422a9595656444f9e984c8f57b783b0/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.125399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c3aa2f2b864edf9ab8ba847dda239d, user_id=0422a9595656444f9e984c8f57b783b0, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.128064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.142262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1183/2379] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:43 2022] PUT /identity/v3/projects/59c3aa2f2b864edf9ab8ba847dda239d/users/0422a9595656444f9e984c8f57b783b0/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:43.152606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8ec31-73d6-4cea-95ec-208acda3015a None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.152893 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8ec31-73d6-4cea-95ec-208acda3015a None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.153236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8ec31-73d6-4cea-95ec-208acda3015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.180899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-62e8ec31-73d6-4cea-95ec-208acda3015a None None] MFA Rules not processed for user `0422a9595656444f9e984c8f57b783b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:43.182156 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-62e8ec31-73d6-4cea-95ec-208acda3015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:43.231923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:43.253326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1197/2380] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:43.258004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.322020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7653f2-cf4f-4a33-aea4-113b17e2a02d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.328115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.329114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.329401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.329677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.330374 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.332328 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.362069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1184/2381] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:43 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:43.366223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8ec31-73d6-4cea-95ec-208acda3015a None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.377780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8ec31-73d6-4cea-95ec-208acda3015a None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.437721 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.438536 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.438826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.439108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.439806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.441784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.466200 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1198/2382] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:43 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:43.467520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.478659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c28a6-ea09-444f-a72b-094a679ba62d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.483675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.484354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.484638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.484910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.485718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.487619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.502373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1185/2383] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:43.506751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.517392 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefbaad-475b-4211-9cd8-75ccc6d7512e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.523250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351cee1-15ee-41e8-9f1a-4c9078ca432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.523929 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351cee1-15ee-41e8-9f1a-4c9078ca432f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.524215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351cee1-15ee-41e8-9f1a-4c9078ca432f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.524493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351cee1-15ee-41e8-9f1a-4c9078ca432f admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.525262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351cee1-15ee-41e8-9f1a-4c9078ca432f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.526912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351cee1-15ee-41e8-9f1a-4c9078ca432f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.538165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1199/2384] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:43.542615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.553449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c897-47f7-445a-916d-0cd5b6301b8c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.558790 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.559663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.559941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.560206 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] PATH_INFO: `/v3/projects/0bcfa16339054a819181e1afccee320f/users/1e9f6a524c3b48ee946085ee6de4fb51/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.576957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcfa16339054a819181e1afccee320f, user_id=1e9f6a524c3b48ee946085ee6de4fb51, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.579779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.594976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1186/2385] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:43 2022] PUT /identity/v3/projects/0bcfa16339054a819181e1afccee320f/users/1e9f6a524c3b48ee946085ee6de4fb51/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:43.606559 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7516cd-2b6c-4c03-898f-a03837efadb8 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.606880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7516cd-2b6c-4c03-898f-a03837efadb8 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.607162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7516cd-2b6c-4c03-898f-a03837efadb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.637883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6d7516cd-2b6c-4c03-898f-a03837efadb8 None None] MFA Rules not processed for user `1e9f6a524c3b48ee946085ee6de4fb51`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:43.639066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6d7516cd-2b6c-4c03-898f-a03837efadb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:43.712234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1200/2386] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:43.716907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.783154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6cb7c-4e6f-4c5a-867c-c7b25f44a00e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.789098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.789761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.790080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.790422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.791084 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.793149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.822436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1187/2387] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:43 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:43.823652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7516cd-2b6c-4c03-898f-a03837efadb8 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.835138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7516cd-2b6c-4c03-898f-a03837efadb8 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.890522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:43.898888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.899577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.899860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.900138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.900845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.902995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.924455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1201/2388] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:43 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:43.928671 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.939188 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274b4e8-c964-4b16-8ab6-736348224b83 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.944092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.944755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.945015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.945289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.946191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.947747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:43.959531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1188/2389] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:43.963778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:43.974246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f324b7-c718-41dd-b888-dec2222c0aff admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:43.978861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:43.979746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:43.980026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:43.980298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] PATH_INFO: `/v3/projects/c7b1b2a1f01c4c4caf65a49504231f99/users/112e6c63403440d2a1b798d601b0b7a1/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:43.995995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b1b2a1f01c4c4caf65a49504231f99, user_id=112e6c63403440d2a1b798d601b0b7a1, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:43.998719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:44.017775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1202/2390] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:43 2022] PUT /identity/v3/projects/c7b1b2a1f01c4c4caf65a49504231f99/users/112e6c63403440d2a1b798d601b0b7a1/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:44.022315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:44.029159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:44.036426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dacad7-2758-4839-bef9-a662bbaf7d5b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:44.096770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa65eb-2ccd-4a65-84bc-52094ba5cfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:44.097476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa65eb-2ccd-4a65-84bc-52094ba5cfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:44.097752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa65eb-2ccd-4a65-84bc-52094ba5cfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:44.098043 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa65eb-2ccd-4a65-84bc-52094ba5cfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:44.098793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa65eb-2ccd-4a65-84bc-52094ba5cfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:44.100398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa65eb-2ccd-4a65-84bc-52094ba5cfb6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:44.110657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1189/2391] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:44.116652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:44.130600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ad5d0-bffe-4db0-ac8f-42e45ade0c1a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:44.135541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:44.136433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:44.136716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:44.136989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] PATH_INFO: `/v3/projects/c7b1b2a1f01c4c4caf65a49504231f99/users/112e6c63403440d2a1b798d601b0b7a1/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:44.139082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b1b2a1f01c4c4caf65a49504231f99, user_id=112e6c63403440d2a1b798d601b0b7a1, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:44.141887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:44.152243 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1203/2392] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:44 2022] PUT /identity/v3/projects/c7b1b2a1f01c4c4caf65a49504231f99/users/112e6c63403440d2a1b798d601b0b7a1/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:44.164513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:44.164945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:44.165440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:44.194501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] MFA Rules not processed for user `112e6c63403440d2a1b798d601b0b7a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:44.195719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:44.266154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1190/2393] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:44.277932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:44.280069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:44.339348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d67fc-fe2f-43ff-a344-205d3b7ece47 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:44.343021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92864bbb-0da2-43de-a193-3fb622ecc6bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:44.343657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92864bbb-0da2-43de-a193-3fb622ecc6bd service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:44.343958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92864bbb-0da2-43de-a193-3fb622ecc6bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:44.344767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92864bbb-0da2-43de-a193-3fb622ecc6bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:44.353950 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92864bbb-0da2-43de-a193-3fb622ecc6bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:44.355389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92864bbb-0da2-43de-a193-3fb622ecc6bd service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:44.364851 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1204/2394] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:44.420169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:44.422343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:44.435626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bee6a6-e71e-450d-8c7f-cba43fd1c50b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:44.440150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:44.440801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:44.441118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:44.441406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:44.499571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:44.501257 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:44.510436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1191/2395] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:44.759189 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b6542-f1fd-4521-ae51-7a839727d5a5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:44.759665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b6542-f1fd-4521-ae51-7a839727d5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:44.759848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b6542-f1fd-4521-ae51-7a839727d5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:44.779343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b52b6542-f1fd-4521-ae51-7a839727d5a5 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:44.780933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b52b6542-f1fd-4521-ae51-7a839727d5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:44.845288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1205/2396] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 07:19:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:44.860767 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:44.863116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:44.894815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:44.951368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9058c-0155-41cc-94d4-14cd409ea46a service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:44.956434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e0ee9-19c5-4af7-97ec-8ca66a2016e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:44.957121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e0ee9-19c5-4af7-97ec-8ca66a2016e8 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:44.957447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e0ee9-19c5-4af7-97ec-8ca66a2016e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:44.957716 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e0ee9-19c5-4af7-97ec-8ca66a2016e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:44.966294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e0ee9-19c5-4af7-97ec-8ca66a2016e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:44.967956 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e0ee9-19c5-4af7-97ec-8ca66a2016e8 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:44.976174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1192/2397] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:19:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:19:47.544502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b6542-f1fd-4521-ae51-7a839727d5a5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:47.612475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:47.619719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b6542-f1fd-4521-ae51-7a839727d5a5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:47.627297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:47.627999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:47.628370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:47.628537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] PATH_INFO: `/v3/users/0422a9595656444f9e984c8f57b783b0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:47.629834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0422a9595656444f9e984c8f57b783b0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:47.631926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:47.700947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0422a9595656444f9e984c8f57b783b0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.703033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0422a9595656444f9e984c8f57b783b0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.703686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0422a9595656444f9e984c8f57b783b0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.718363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0422a9595656444f9e984c8f57b783b0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.730274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0422a9595656444f9e984c8f57b783b0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.751129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1206/2398] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:47 2022] DELETE /identity/v3/users/0422a9595656444f9e984c8f57b783b0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:47.756289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:47.756779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:47.756884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:47.758731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:47.760182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:47.777193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:47.778689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:47.835195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8393fe-3e6d-4c62-80fe-42030be4c4a0 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:47.840590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:47.842073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:47.842814 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:47.843158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] PATH_INFO: `/v3/users/2195beda0ccd46a7acd8652f39d98927` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:47.844458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2195beda0ccd46a7acd8652f39d98927)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:47.846478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:47.851070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1193/2399] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:47.864000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:47.865639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:47.902012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2195beda0ccd46a7acd8652f39d98927'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.902874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2195beda0ccd46a7acd8652f39d98927'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.903479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2195beda0ccd46a7acd8652f39d98927'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.920664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2195beda0ccd46a7acd8652f39d98927'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.928372 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1041d09b-41ed-4025-aca1-3efe1576f909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2195beda0ccd46a7acd8652f39d98927'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:47.931450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c07b3-fa15-43bd-9f33-84bd2cda2fac None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:47.931587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:47.936314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:47.937216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:47.937379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:47.937780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:47.939526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:47.949531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1207/2400] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:47 2022] DELETE /identity/v3/users/2195beda0ccd46a7acd8652f39d98927 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:47.954514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:47.954804 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:47.955176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:47.955865 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:47.956715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:47.967908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1194/2401] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:47.975221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:47.976452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:48.049529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1208/2402] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:48.059818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:48.061878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:48.074004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5db418-9e6f-40fa-9899-f52861c62e25 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:48.141125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:48.141897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:48.142391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:48.142889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:48.154731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:48.156683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:48.161120 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:48.173371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1195/2403] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:48.339770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:48.341714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:48.354424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebc417-c1ef-44f5-9692-11f7b00ae380 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:48.359869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:48.360735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:48.361009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:48.361400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] PATH_INFO: `/v3/projects/59c3aa2f2b864edf9ab8ba847dda239d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:48.362651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c3aa2f2b864edf9ab8ba847dda239d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:48.364838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:48.445300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c3aa2f2b864edf9ab8ba847dda239d', 'request_id': 'req-86801d3c-b078-4433-ada1-35b27e46aedb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.452302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c3aa2f2b864edf9ab8ba847dda239d', 'request_id': 'req-86801d3c-b078-4433-ada1-35b27e46aedb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.457922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c3aa2f2b864edf9ab8ba847dda239d', 'request_id': 'req-86801d3c-b078-4433-ada1-35b27e46aedb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.474040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] The token cache is being invalidate because project 59c3aa2f2b864edf9ab8ba847dda239d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:48.474303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.474945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.476522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1209/2404] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:48 2022] DELETE /identity/v3/projects/59c3aa2f2b864edf9ab8ba847dda239d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:48.480546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:48.490770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870f65b-312a-430d-a5de-763369db0534 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:48.516132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:48.517591 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:48.555666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:48.556565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:48.556843 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:48.557147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] PATH_INFO: `/v3/users/1e9f6a524c3b48ee946085ee6de4fb51` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:48.558340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9f6a524c3b48ee946085ee6de4fb51)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:48.560336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:48.599545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86801d3c-b078-4433-ada1-35b27e46aedb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:48.604408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:48.605314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:48.605611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:48.605887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] PATH_INFO: `/v3/projects/63fe07e5855349b8aff8d8a68c37e649` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:48.607203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fe07e5855349b8aff8d8a68c37e649)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:48.608488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9f6a524c3b48ee946085ee6de4fb51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.609146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9f6a524c3b48ee946085ee6de4fb51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.609254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:48.622036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9f6a524c3b48ee946085ee6de4fb51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.622638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9f6a524c3b48ee946085ee6de4fb51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.629531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9f6a524c3b48ee946085ee6de4fb51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.630977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:48.644083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1196/2405] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:48 2022] DELETE /identity/v3/users/1e9f6a524c3b48ee946085ee6de4fb51 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:48.661502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fe07e5855349b8aff8d8a68c37e649', 'request_id': 'req-4eb5393a-9269-4fed-979c-de37ec24c143'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.667936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fe07e5855349b8aff8d8a68c37e649', 'request_id': 'req-4eb5393a-9269-4fed-979c-de37ec24c143'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.673492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fe07e5855349b8aff8d8a68c37e649', 'request_id': 'req-4eb5393a-9269-4fed-979c-de37ec24c143'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.687136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] The token cache is being invalidate because project 63fe07e5855349b8aff8d8a68c37e649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:48.687441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.688085 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.689696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1210/2406] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:48 2022] DELETE /identity/v3/projects/63fe07e5855349b8aff8d8a68c37e649 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:48.694354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:48.695848 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:48.763674 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e096c-002c-4108-9a9d-9c3b3d2da8c7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:48.784958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:48.834607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:48.835643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:48.835953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:48.836237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] PATH_INFO: `/v3/users/00dd854ed9e940cba222b953da5a4167` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:48.837901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00dd854ed9e940cba222b953da5a4167)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:48.839971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:48.892886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00dd854ed9e940cba222b953da5a4167'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.893595 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00dd854ed9e940cba222b953da5a4167'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.907047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00dd854ed9e940cba222b953da5a4167'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.907721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00dd854ed9e940cba222b953da5a4167'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.915299 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00dd854ed9e940cba222b953da5a4167'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:48.928648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1197/2407] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:48 2022] DELETE /identity/v3/users/00dd854ed9e940cba222b953da5a4167 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:48.933083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:48.934519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:48.953408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5393a-9269-4fed-979c-de37ec24c143 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.021478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.022504 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.022784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.023159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] PATH_INFO: `/v3/projects/0bcfa16339054a819181e1afccee320f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.024577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcfa16339054a819181e1afccee320f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.026798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.079598 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcfa16339054a819181e1afccee320f', 'request_id': 'req-42568a96-02a8-4805-b676-4f8687f84b0d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.086756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcfa16339054a819181e1afccee320f', 'request_id': 'req-42568a96-02a8-4805-b676-4f8687f84b0d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.093594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcfa16339054a819181e1afccee320f', 'request_id': 'req-42568a96-02a8-4805-b676-4f8687f84b0d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.111033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] The token cache is being invalidate because project 0bcfa16339054a819181e1afccee320f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:49.111391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.112107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.113919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1211/2408] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:48 2022] DELETE /identity/v3/projects/0bcfa16339054a819181e1afccee320f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:49.118006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.119404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.132655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413013f1-2bcf-4bb7-9084-6c45938b203f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.191461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.192400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.192837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.193036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] PATH_INFO: `/v3/users/112e6c63403440d2a1b798d601b0b7a1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.194278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=112e6c63403440d2a1b798d601b0b7a1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.196205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.236275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.238370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.255709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42568a96-02a8-4805-b676-4f8687f84b0d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.256009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112e6c63403440d2a1b798d601b0b7a1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.256671 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112e6c63403440d2a1b798d601b0b7a1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.258999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:49.263239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.264139 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.264425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.265029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] PATH_INFO: `/v3/projects/eff6d755352648559e5fde0abc35e46e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.267649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff6d755352648559e5fde0abc35e46e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.270219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.279635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112e6c63403440d2a1b798d601b0b7a1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.280677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112e6c63403440d2a1b798d601b0b7a1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.289919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f03f16fd-ea72-47c6-bd2c-2a9d9d7e3037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112e6c63403440d2a1b798d601b0b7a1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.307741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1198/2409] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:49 2022] DELETE /identity/v3/users/112e6c63403440d2a1b798d601b0b7a1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:49.315327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:49.331203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff6d755352648559e5fde0abc35e46e', 'request_id': 'req-cef975a3-ee87-42c7-bf78-d0c6b5f56214'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.337959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff6d755352648559e5fde0abc35e46e', 'request_id': 'req-cef975a3-ee87-42c7-bf78-d0c6b5f56214'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.343470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff6d755352648559e5fde0abc35e46e', 'request_id': 'req-cef975a3-ee87-42c7-bf78-d0c6b5f56214'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.357099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] The token cache is being invalidate because project eff6d755352648559e5fde0abc35e46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:49.357460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.358277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-cef975a3-ee87-42c7-bf78-d0c6b5f56214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.360608 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1212/2410] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:49 2022] DELETE /identity/v3/projects/eff6d755352648559e5fde0abc35e46e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:49.367077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785269f0-cf9a-4069-8838-02b1732bc3a4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.367365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785269f0-cf9a-4069-8838-02b1732bc3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.367639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785269f0-cf9a-4069-8838-02b1732bc3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.388413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-785269f0-cf9a-4069-8838-02b1732bc3a4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:49.389680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-785269f0-cf9a-4069-8838-02b1732bc3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.450000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1199/2411] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:49.458540 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f8db-ef26-4fc9-a035-6b7f6da40968 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.459000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f8db-ef26-4fc9-a035-6b7f6da40968 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.459452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49f8db-ef26-4fc9-a035-6b7f6da40968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.480423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2a49f8db-ef26-4fc9-a035-6b7f6da40968 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:49.481670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2a49f8db-ef26-4fc9-a035-6b7f6da40968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.489639 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1213/2412] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:49.494138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785269f0-cf9a-4069-8838-02b1732bc3a4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.503955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785269f0-cf9a-4069-8838-02b1732bc3a4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.508764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.509506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.509809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.510066 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.510805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:49.511116 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.512720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.524786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1200/2413] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:19:49.529255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f8db-ef26-4fc9-a035-6b7f6da40968 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.539100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49f8db-ef26-4fc9-a035-6b7f6da40968 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.544305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.544635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.544914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.545220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.545907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.547782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.578411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1214/2414] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:49.583177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.588074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.589956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.595183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0aeffc-4dc9-49f9-b22d-32895a9aea5e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.659004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc43788-63cd-4399-a545-ed63b1aeb378 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.661400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.662144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.662426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.662702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.663400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.664791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.665360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.665702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.665991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.666266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] PATH_INFO: `/v3/projects/c7b1b2a1f01c4c4caf65a49504231f99` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.667443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b1b2a1f01c4c4caf65a49504231f99)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.669365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.686531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1201/2415] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:49.690652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.700665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69ef6-4596-4730-b3dc-60738dded8fe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.705371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.706031 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.706311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.706585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.707312 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.709571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.719492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b1b2a1f01c4c4caf65a49504231f99', 'request_id': 'req-6b2d0146-dffd-465f-ac5d-95a6c45031ad'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.720240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1202/2416] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:49.724289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.725871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b1b2a1f01c4c4caf65a49504231f99', 'request_id': 'req-6b2d0146-dffd-465f-ac5d-95a6c45031ad'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.731615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b1b2a1f01c4c4caf65a49504231f99', 'request_id': 'req-6b2d0146-dffd-465f-ac5d-95a6c45031ad'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.734356 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fed0d8-e757-45f1-a2dd-2ec281feec70 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.746298 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] The token cache is being invalidate because project c7b1b2a1f01c4c4caf65a49504231f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:49.746584 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.747218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-6b2d0146-dffd-465f-ac5d-95a6c45031ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:49.748790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1215/2417] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:49 2022] DELETE /identity/v3/projects/c7b1b2a1f01c4c4caf65a49504231f99 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:49.760436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75769e-7aa2-4a11-84b8-428659308602 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.760718 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75769e-7aa2-4a11-84b8-428659308602 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.760991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75769e-7aa2-4a11-84b8-428659308602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.763087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:49.780992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ea75769e-7aa2-4a11-84b8-428659308602 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:49.782291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ea75769e-7aa2-4a11-84b8-428659308602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.796926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.797634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.797917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.798190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.798911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.799899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:49.801221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.811181 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1203/2418] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:49.815370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.817248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.830517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d6cd-075f-4332-b6ec-5490af3a74e0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.835519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc045f-5839-4664-b6f5-f2a510231040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.836407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc045f-5839-4664-b6f5-f2a510231040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.836679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc045f-5839-4664-b6f5-f2a510231040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.836944 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc045f-5839-4664-b6f5-f2a510231040 admin admin] PATH_INFO: `/v3/projects/98b3be09c53a4c6e86397e421b1e29fe/users/69d5a191ccd44bb1947ae7709ea0f2ee/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.847485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1216/2419] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:49.854602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc045f-5839-4664-b6f5-f2a510231040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b3be09c53a4c6e86397e421b1e29fe, user_id=69d5a191ccd44bb1947ae7709ea0f2ee, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.856132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1685bd-209a-4bcf-8f26-7d0938668089 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.856490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1685bd-209a-4bcf-8f26-7d0938668089 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.856841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1685bd-209a-4bcf-8f26-7d0938668089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.856984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc045f-5839-4664-b6f5-f2a510231040 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.872239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1204/2420] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:49 2022] PUT /identity/v3/projects/98b3be09c53a4c6e86397e421b1e29fe/users/69d5a191ccd44bb1947ae7709ea0f2ee/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:49.876166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8d1685bd-209a-4bcf-8f26-7d0938668089 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:49.877752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8d1685bd-209a-4bcf-8f26-7d0938668089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.884633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9229f-01c5-42f5-ab01-5cf8cf3704e6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.884916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9229f-01c5-42f5-ab01-5cf8cf3704e6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.885289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9229f-01c5-42f5-ab01-5cf8cf3704e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.912250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-dcd9229f-01c5-42f5-ab01-5cf8cf3704e6 None None] MFA Rules not processed for user `69d5a191ccd44bb1947ae7709ea0f2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:49.913622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dcd9229f-01c5-42f5-ab01-5cf8cf3704e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:49.937711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1217/2421] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:49.942004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1685bd-209a-4bcf-8f26-7d0938668089 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.950916 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1685bd-209a-4bcf-8f26-7d0938668089 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.955697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.956339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.956627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.956900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.957658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:49.957964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.959709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.971354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1218/2422] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:19:49.975648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.978949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1205/2423] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:49.983456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9229f-01c5-42f5-ab01-5cf8cf3704e6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:49.985669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecbde9-15af-4435-aae6-5e1af9a7beab admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:49.990667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:49.991311 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:49.991596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:49.991874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:49.992562 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:49.994628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:49.997569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9229f-01c5-42f5-ab01-5cf8cf3704e6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.002587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.003307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.003588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.003861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.004549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.006508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.022423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1219/2424] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:50.028276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.036875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1206/2425] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:49 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:50.040827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.090317 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ac1b3-2087-4417-bbb5-c10bb7f4369b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.102938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c55e79-ed5c-4791-9440-365dba3d9af2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.107448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.108121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.108395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.108663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.109373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.111285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.122621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:50.128072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:50.139407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1207/2426] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:50 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:50.140186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.150457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ad90e-ef07-4507-a407-6b7eb78ec683 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.150745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.151427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.151704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.151975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.152661 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.154627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.155683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.156348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.156632 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.156907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.157670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.159305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.171129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1208/2427] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:50.175252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.179015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1220/2428] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:50 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:50.180023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.185904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5f77-3469-4048-892b-176dd8c7d663 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.189568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f196f-2961-4e6a-989e-1d13f5da49dd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.191183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.192048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.192330 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.192757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] PATH_INFO: `/v3/projects/0fc6d31207224e41a2e6c2e95e09af03/users/a6722bf0ab394e588ced846a280a8096/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.194520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.195159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.195436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.195708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.196487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.198068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.208169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1221/2429] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:50.208644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc6d31207224e41a2e6c2e95e09af03, user_id=a6722bf0ab394e588ced846a280a8096, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.211497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.212237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.222395 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367ba9e-2684-4547-8f3c-a936e761f25e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.226285 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1209/2430] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:50 2022] PUT /identity/v3/projects/0fc6d31207224e41a2e6c2e95e09af03/users/a6722bf0ab394e588ced846a280a8096/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:50.230115 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.280193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.281251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.281533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.281805 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.282507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.284117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.291223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b72da9-602d-4ace-a89d-8125f8458eab admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.294026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1222/2431] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:50.296165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.296826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.297132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.297392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.298117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.298443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.299705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.311179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5667d0-7b01-496e-88d1-85667593710d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.311323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1210/2432] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:50.314511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.318182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94f5c6-6585-4cfc-8db6-99dbf1190cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.319325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f5c6-6585-4cfc-8db6-99dbf1190cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.319892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f5c6-6585-4cfc-8db6-99dbf1190cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.320305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f5c6-6585-4cfc-8db6-99dbf1190cbe admin admin] PATH_INFO: `/v3/projects/0c4dc993f0d343b7baabd8f7e93bae80/users/436ef3977f3c49afaa343bf2a4c98210/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.325544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71a83f-2910-4ea1-a730-a23e46c31051 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.330450 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12323af0-d812-4b66-91b8-e6f9c0220510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.331363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323af0-d812-4b66-91b8-e6f9c0220510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.331626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323af0-d812-4b66-91b8-e6f9c0220510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.331896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323af0-d812-4b66-91b8-e6f9c0220510 admin admin] PATH_INFO: `/v3/projects/0fc6d31207224e41a2e6c2e95e09af03/users/a6722bf0ab394e588ced846a280a8096/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.333921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12323af0-d812-4b66-91b8-e6f9c0220510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc6d31207224e41a2e6c2e95e09af03, user_id=a6722bf0ab394e588ced846a280a8096, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.336543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12323af0-d812-4b66-91b8-e6f9c0220510 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.337683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94f5c6-6585-4cfc-8db6-99dbf1190cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4dc993f0d343b7baabd8f7e93bae80, user_id=436ef3977f3c49afaa343bf2a4c98210, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.340334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94f5c6-6585-4cfc-8db6-99dbf1190cbe admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.347451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1211/2433] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:50 2022] PUT /identity/v3/projects/0fc6d31207224e41a2e6c2e95e09af03/users/a6722bf0ab394e588ced846a280a8096/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:50.362787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1223/2434] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:50 2022] PUT /identity/v3/projects/0c4dc993f0d343b7baabd8f7e93bae80/users/436ef3977f3c49afaa343bf2a4c98210/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:50.363009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.363009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.363009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.368903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0991cf-4d47-4f12-9573-8d14eddd2987 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.369307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0991cf-4d47-4f12-9573-8d14eddd2987 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.369670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0991cf-4d47-4f12-9573-8d14eddd2987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.392194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] MFA Rules not processed for user `a6722bf0ab394e588ced846a280a8096`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:50.393440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:50.399325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0f0991cf-4d47-4f12-9573-8d14eddd2987 None None] MFA Rules not processed for user `436ef3977f3c49afaa343bf2a4c98210`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:50.400615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0f0991cf-4d47-4f12-9573-8d14eddd2987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:50.465324 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1224/2435] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:50.468075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1212/2436] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:50.469869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0991cf-4d47-4f12-9573-8d14eddd2987 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.479664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.481028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:50.487325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:50.517846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:50.538902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0991cf-4d47-4f12-9573-8d14eddd2987 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.543507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.544177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.544455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.544724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.545447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.546989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cec99-e80c-4151-9da2-15fb46866e70 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.547347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.550869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.551524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.551804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.552272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.562525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.564380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.576985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1225/2437] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:50 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:50.577238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1213/2438] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:19:50.578037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.619814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.621290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:50.632732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3cd2a-d6d5-4a72-a86c-e2b377a5f7b3 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.640371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f864f9-c9ad-4bd3-8cb8-a9ace1059380 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.645390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.646120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.646394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.646666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.647350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.649278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.670789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1226/2439] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:50 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:50.674635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.684799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5527112-32e7-4f71-996f-6144cbf8edfb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.686013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.686709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.686990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.687264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.689388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.690086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.690367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.690638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.691361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.693008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.706169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1227/2440] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:50.707253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.717269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e39db-4f82-4c0c-9231-aabafbc54fd2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.722078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.722966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.723221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.723510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] PATH_INFO: `/v3/projects/ed32c79fb7744102afd921aca8910bad/users/c3e1862ff8814a3b8ac0a49f337812f4/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.738498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed32c79fb7744102afd921aca8910bad, user_id=c3e1862ff8814a3b8ac0a49f337812f4, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.741458 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.744869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.746597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.754897 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1228/2441] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:50 2022] PUT /identity/v3/projects/ed32c79fb7744102afd921aca8910bad/users/c3e1862ff8814a3b8ac0a49f337812f4/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:50.758922 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.759543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1214/2442] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:19:50.819158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:50.824756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449438-7271-461e-9953-5d07077e6fda admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.830449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a9d50-4f4d-4448-8aa9-15e10795545b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.831127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a9d50-4f4d-4448-8aa9-15e10795545b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.831403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a9d50-4f4d-4448-8aa9-15e10795545b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.831678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a9d50-4f4d-4448-8aa9-15e10795545b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.832402 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a9d50-4f4d-4448-8aa9-15e10795545b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.834305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a9d50-4f4d-4448-8aa9-15e10795545b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.844578 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1229/2443] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:50.849517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:50.860502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b213f4f-49ea-4612-b5ad-dad6b996227c service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:50.865258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:50.866151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.866429 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.866698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] PATH_INFO: `/v3/projects/ed32c79fb7744102afd921aca8910bad/users/c3e1862ff8814a3b8ac0a49f337812f4/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.868740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed32c79fb7744102afd921aca8910bad, user_id=c3e1862ff8814a3b8ac0a49f337812f4, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:50.871565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:50.884223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1215/2444] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:50 2022] PUT /identity/v3/projects/ed32c79fb7744102afd921aca8910bad/users/c3e1862ff8814a3b8ac0a49f337812f4/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:50.895858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e78e6-98a3-4769-a889-21d05eebedd4 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:50.896023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e78e6-98a3-4769-a889-21d05eebedd4 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:50.896235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e78e6-98a3-4769-a889-21d05eebedd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:50.924583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2c1e78e6-98a3-4769-a889-21d05eebedd4 None None] MFA Rules not processed for user `c3e1862ff8814a3b8ac0a49f337812f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:50.925838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2c1e78e6-98a3-4769-a889-21d05eebedd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:51.004864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1230/2445] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:19:51.017911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:51.080406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1a751-9527-45ca-8926-783e3454b79c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:51.084385 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:51.085082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:51.085375 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:51.085726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:51.095073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:51.096517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:51.106663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1216/2446] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:51.191772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e78e6-98a3-4769-a889-21d05eebedd4 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:51.203319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e78e6-98a3-4769-a889-21d05eebedd4 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:51.207669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe4288-d4cb-4d7f-91c6-b36064199292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:51.208339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe4288-d4cb-4d7f-91c6-b36064199292 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:51.208614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe4288-d4cb-4d7f-91c6-b36064199292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:51.208883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe4288-d4cb-4d7f-91c6-b36064199292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:51.266891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe4288-d4cb-4d7f-91c6-b36064199292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:51.268338 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe4288-d4cb-4d7f-91c6-b36064199292 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:51.279833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1231/2447] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:51.559641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:51.580015 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:51.624032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497e49-17c3-4dc3-b732-c9be8d7fb8e4 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:51.628909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:51.629825 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:51.630143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:51.630483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] PATH_INFO: `/v3/users/436ef3977f3c49afaa343bf2a4c98210` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:51.631808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] RBAC: Authorizing `identity:delete_user(user_id=436ef3977f3c49afaa343bf2a4c98210)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:51.633750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:51.684225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436ef3977f3c49afaa343bf2a4c98210'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:51.684873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436ef3977f3c49afaa343bf2a4c98210'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:51.697837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436ef3977f3c49afaa343bf2a4c98210'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:51.698493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436ef3977f3c49afaa343bf2a4c98210'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:51.705435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436ef3977f3c49afaa343bf2a4c98210'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:51.720056 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1217/2448] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:51 2022] DELETE /identity/v3/users/436ef3977f3c49afaa343bf2a4c98210 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:51.724876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:51.725396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:51.725511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:51.743751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:51.744952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:51.809349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1232/2449] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:51.819335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:51.820810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:51.878793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9beb47-bb2e-4726-9fce-72c3c253b64c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:51.882910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:51.883556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:51.883898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:51.884171 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:51.893553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:51.894966 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:51.904294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1218/2450] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:19:52.177244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:52.178669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:52.195567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f524cb-36a9-4353-8d17-404232e34108 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:52.202944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:52.204368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:52.204935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:52.205473 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] PATH_INFO: `/v3/projects/0c4dc993f0d343b7baabd8f7e93bae80` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:52.207071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4dc993f0d343b7baabd8f7e93bae80)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:52.209343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:52.221474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:52.261286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4dc993f0d343b7baabd8f7e93bae80', 'request_id': 'req-1e12745d-10c9-4e71-9170-0a07e147eeda'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.267875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4dc993f0d343b7baabd8f7e93bae80', 'request_id': 'req-1e12745d-10c9-4e71-9170-0a07e147eeda'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.274109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4dc993f0d343b7baabd8f7e93bae80', 'request_id': 'req-1e12745d-10c9-4e71-9170-0a07e147eeda'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.288406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] The token cache is being invalidate because project 0c4dc993f0d343b7baabd8f7e93bae80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:52.288699 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.289341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.291106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1233/2451] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:52 2022] DELETE /identity/v3/projects/0c4dc993f0d343b7baabd8f7e93bae80 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:52.295016 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:52.304828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d4e5f-ab4c-44eb-a5fa-f69687a42fb7 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:52.343473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:52.363797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:52.365154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:52.365460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:52.365721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] PATH_INFO: `/v3/users/c3e1862ff8814a3b8ac0a49f337812f4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:52.366942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e1862ff8814a3b8ac0a49f337812f4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:52.368927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:52.419888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e1862ff8814a3b8ac0a49f337812f4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.421322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e1862ff8814a3b8ac0a49f337812f4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.434300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e1862ff8814a3b8ac0a49f337812f4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.435177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e1862ff8814a3b8ac0a49f337812f4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.443770 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e301e24-d894-4ae6-a822-25cad4ee37c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e1862ff8814a3b8ac0a49f337812f4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.478856 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1219/2452] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:52 2022] DELETE /identity/v3/users/c3e1862ff8814a3b8ac0a49f337812f4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:52.774345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:52.775951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:52.794566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12745d-10c9-4e71-9170-0a07e147eeda admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:52.859483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:52.860369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:52.860653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:52.860931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] PATH_INFO: `/v3/projects/ed32c79fb7744102afd921aca8910bad` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:52.862168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed32c79fb7744102afd921aca8910bad)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:52.864137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:52.919486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed32c79fb7744102afd921aca8910bad', 'request_id': 'req-346b76cb-f320-4ae1-89ab-4b6c451efa37'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.925949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed32c79fb7744102afd921aca8910bad', 'request_id': 'req-346b76cb-f320-4ae1-89ab-4b6c451efa37'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.931819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed32c79fb7744102afd921aca8910bad', 'request_id': 'req-346b76cb-f320-4ae1-89ab-4b6c451efa37'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.945617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] The token cache is being invalidate because project ed32c79fb7744102afd921aca8910bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:52.945904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.946520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-346b76cb-f320-4ae1-89ab-4b6c451efa37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:52.948129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1234/2453] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:52 2022] DELETE /identity/v3/projects/ed32c79fb7744102afd921aca8910bad => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:52.954358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f95cc-d55b-4c38-822a-ee1d6987ed3c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:52.954838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f95cc-d55b-4c38-822a-ee1d6987ed3c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:52.954954 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f95cc-d55b-4c38-822a-ee1d6987ed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:52.974108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ea1f95cc-d55b-4c38-822a-ee1d6987ed3c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:52.975327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ea1f95cc-d55b-4c38-822a-ee1d6987ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:53.038601 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1220/2454] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:53.047801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795245f-9400-42b8-b878-ebb053195cf5 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.048090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795245f-9400-42b8-b878-ebb053195cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.048385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795245f-9400-42b8-b878-ebb053195cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.069576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1795245f-9400-42b8-b878-ebb053195cf5 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:53.070792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1795245f-9400-42b8-b878-ebb053195cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:53.080248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1235/2455] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:53.084982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f95cc-d55b-4c38-822a-ee1d6987ed3c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.095110 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f95cc-d55b-4c38-822a-ee1d6987ed3c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.100437 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.101090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.101400 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.101695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.102423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:53.102729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.104313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.116210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1221/2456] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:19:53.120701 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795245f-9400-42b8-b878-ebb053195cf5 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.131010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795245f-9400-42b8-b878-ebb053195cf5 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.136423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.137136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.137491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.137766 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.138517 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.140394 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.164135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:53.173149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1236/2457] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:53 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:53.178223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.188694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f04cc7-5e4e-4643-867e-f6612112730f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.239198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:53.256575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.257335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.257615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.257888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.258588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.260509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.282914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1222/2458] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:53 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:53.288979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.305003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b29208-dddd-4380-beb2-4e609a3631f4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.311570 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.312244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.312525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.312822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.313575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.315220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.326255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1237/2459] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:53.330382 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.340809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98349f-7c40-4923-951b-3fc1b8c1f87a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.346023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d282d-fa22-45c9-816e-887fba9f700b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.346683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d282d-fa22-45c9-816e-887fba9f700b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.346958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d282d-fa22-45c9-816e-887fba9f700b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.347230 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d282d-fa22-45c9-816e-887fba9f700b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.347940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d282d-fa22-45c9-816e-887fba9f700b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.349653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d282d-fa22-45c9-816e-887fba9f700b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.359799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1223/2460] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:53.364016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.374603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418f697-538a-4901-a5bc-1c013fa7dfd7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.381259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.382144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.382418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.382683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] PATH_INFO: `/v3/projects/b57b978a818a4f1bb7caaab37bb46f45/users/2eef71100dcc45caa7f13d362777eb38/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.398564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57b978a818a4f1bb7caaab37bb46f45, user_id=2eef71100dcc45caa7f13d362777eb38, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.401257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.417516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1238/2461] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:53 2022] PUT /identity/v3/projects/b57b978a818a4f1bb7caaab37bb46f45/users/2eef71100dcc45caa7f13d362777eb38/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:53.428304 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bca36-2e8b-45fd-8c93-906149336895 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.428594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bca36-2e8b-45fd-8c93-906149336895 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.428874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bca36-2e8b-45fd-8c93-906149336895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.455932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c34bca36-2e8b-45fd-8c93-906149336895 None None] MFA Rules not processed for user `2eef71100dcc45caa7f13d362777eb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:53.457130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c34bca36-2e8b-45fd-8c93-906149336895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:53.533900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1224/2462] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:53.538744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.612428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bfff6-e208-4487-80aa-82a36345f077 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.617274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.617986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.618203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.618472 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.619150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.621026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.650890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1239/2463] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:53 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:53.652224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bca36-2e8b-45fd-8c93-906149336895 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.664967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bca36-2e8b-45fd-8c93-906149336895 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.722529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.723212 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.723489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.723816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.724491 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.726441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.747271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1225/2464] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:53 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:53.751932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.762332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196009a4-bb4d-4526-8506-8c10847f7d19 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.768498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.769221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.769626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.769907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.770807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.773149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.792671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1240/2465] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:53.797288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.807610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f8491-9e22-4aee-bb15-0d46bf736453 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.812731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02027e-f837-48a5-98a9-389dd57ec0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.813537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02027e-f837-48a5-98a9-389dd57ec0cf admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.813814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02027e-f837-48a5-98a9-389dd57ec0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.814081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02027e-f837-48a5-98a9-389dd57ec0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.814809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02027e-f837-48a5-98a9-389dd57ec0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.816415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02027e-f837-48a5-98a9-389dd57ec0cf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.829147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1226/2466] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:53.831428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:53.842077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63a80a-7588-47dc-a530-1d24a20f1b3b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:53.847692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:53.848921 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.849343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.849669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] PATH_INFO: `/v3/projects/0187c8f8308c42a6b3ea2566c9c32d9e/users/f3dddc2411da4cdfaf855f694105f766/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.868429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0187c8f8308c42a6b3ea2566c9c32d9e, user_id=f3dddc2411da4cdfaf855f694105f766, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:53.871115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:53.885432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1241/2467] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:53 2022] PUT /identity/v3/projects/0187c8f8308c42a6b3ea2566c9c32d9e/users/f3dddc2411da4cdfaf855f694105f766/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:53.895838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a1277-338d-4406-8f96-a0e303465799 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:53.896224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a1277-338d-4406-8f96-a0e303465799 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:53.896502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a1277-338d-4406-8f96-a0e303465799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:53.925151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-995a1277-338d-4406-8f96-a0e303465799 None None] MFA Rules not processed for user `f3dddc2411da4cdfaf855f694105f766`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:53.926370 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-995a1277-338d-4406-8f96-a0e303465799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:53.948317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:54.009078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1227/2468] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:54.013572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.080666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24710861-f5c3-4331-9c4c-594cd9d4d470 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.086516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.087236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.087567 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.087778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.088474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.090445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.098512 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a1277-338d-4406-8f96-a0e303465799 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.100201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-995a1277-338d-4406-8f96-a0e303465799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.111417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:54.114502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a1277-338d-4406-8f96-a0e303465799 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.119020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.119903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.120183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.120456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] PATH_INFO: `/v3/users/69d5a191ccd44bb1947ae7709ea0f2ee` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.121360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1242/2469] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:54 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:54.121685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=69d5a191ccd44bb1947ae7709ea0f2ee)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.123058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.123628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.172020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69d5a191ccd44bb1947ae7709ea0f2ee'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.172682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69d5a191ccd44bb1947ae7709ea0f2ee'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.185948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69d5a191ccd44bb1947ae7709ea0f2ee'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.186646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69d5a191ccd44bb1947ae7709ea0f2ee'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.187177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f19ea5-131f-48ce-aaa1-f2c4592b1bb1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.191870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.192686 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.192974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.193349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.194082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.194242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9a71f68c-787b-4a4f-94ee-8ee8c02d4acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69d5a191ccd44bb1947ae7709ea0f2ee'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.196014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.207253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1228/2470] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:54 2022] DELETE /identity/v3/users/69d5a191ccd44bb1947ae7709ea0f2ee => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:54.211832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.212121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.212399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.219294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1243/2471] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:54 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:54.220296 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.221933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.231280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:54.232458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.234218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3106-5bc0-4b25-9657-2656a94e0b81 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.293778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.294551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.294833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.294947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1229/2472] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:54.295107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.295898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.297549 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.304906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.306541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.307738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1244/2473] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:54.311635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.321075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166225f-b038-4a36-b69b-7380a6894d14 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.325498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.326359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.326706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.326911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] PATH_INFO: `/v3/projects/75d73d45e3524dc6945d6fecfcfac516/users/63a117dbb0f449df92d39125726c1d87/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.343104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d73d45e3524dc6945d6fecfcfac516, user_id=63a117dbb0f449df92d39125726c1d87, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.345807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.352352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:54.362081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1245/2474] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:54 2022] PUT /identity/v3/projects/75d73d45e3524dc6945d6fecfcfac516/users/63a117dbb0f449df92d39125726c1d87/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:54.366350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.379732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6a061-6fc6-4eba-b660-246d01d64060 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.435568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3dabe-1e20-4617-9325-cb66bd720338 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.439442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e40fb-392a-4bfc-abca-860699d81942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.440101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e40fb-392a-4bfc-abca-860699d81942 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.440406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e40fb-392a-4bfc-abca-860699d81942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.440541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.440682 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e40fb-392a-4bfc-abca-860699d81942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.441098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.441385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.441660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.442420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.444032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.446184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:54.451118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e40fb-392a-4bfc-abca-860699d81942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.452736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e40fb-392a-4bfc-abca-860699d81942 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.458543 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1246/2475] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:54.461531 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.464055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1230/2476] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:54.471703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b13c-c7cd-4f6e-a10a-cf4421ae597d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.476820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.477740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.478019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.478292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] PATH_INFO: `/v3/projects/75d73d45e3524dc6945d6fecfcfac516/users/63a117dbb0f449df92d39125726c1d87/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.480325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d73d45e3524dc6945d6fecfcfac516, user_id=63a117dbb0f449df92d39125726c1d87, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.482976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.494060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1247/2477] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:54 2022] PUT /identity/v3/projects/75d73d45e3524dc6945d6fecfcfac516/users/63a117dbb0f449df92d39125726c1d87/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:54.564588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.564879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.565192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.595097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] MFA Rules not processed for user `63a117dbb0f449df92d39125726c1d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:54.596238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.686568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1231/2478] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:54.699900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.702739 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.771003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b09fe5-dda1-4697-9d64-d4c912fc2c23 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.774777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.775096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.775436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.775712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.775984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.776766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.777974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:54.841276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.842795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.852837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1248/2479] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:54.861814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214c753-771a-44d3-be6e-e42283cfc6b2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:54.867113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:54.868349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:54.868706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:54.869041 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] PATH_INFO: `/v3/projects/98b3be09c53a4c6e86397e421b1e29fe` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:54.870316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b3be09c53a4c6e86397e421b1e29fe)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:54.872301 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:54.903038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:54.933935 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b3be09c53a4c6e86397e421b1e29fe', 'request_id': 'req-5c71f2c8-3a43-4d40-90c1-8f556177df9d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.942818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b3be09c53a4c6e86397e421b1e29fe', 'request_id': 'req-5c71f2c8-3a43-4d40-90c1-8f556177df9d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.949496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b3be09c53a4c6e86397e421b1e29fe', 'request_id': 'req-5c71f2c8-3a43-4d40-90c1-8f556177df9d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.960511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] The token cache is being invalidate because project 98b3be09c53a4c6e86397e421b1e29fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:54.960903 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.961625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:54.963299 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1232/2480] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:54 2022] DELETE /identity/v3/projects/98b3be09c53a4c6e86397e421b1e29fe => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:54.967247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:54.979532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829a205-8e6f-4668-9d50-6e02d5f625d2 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:55.042974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:55.043862 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:55.044135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:55.044406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] PATH_INFO: `/v3/users/a6722bf0ab394e588ced846a280a8096` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:55.045625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6722bf0ab394e588ced846a280a8096)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:55.047600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:55.098061 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:55.105706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6722bf0ab394e588ced846a280a8096'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.106770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6722bf0ab394e588ced846a280a8096'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.108138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6722bf0ab394e588ced846a280a8096'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.124237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6722bf0ab394e588ced846a280a8096'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.142198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-90fd9496-2778-49c9-bebe-d6689aee0090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6722bf0ab394e588ced846a280a8096'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.161302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1249/2481] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:19:54 2022] DELETE /identity/v3/users/a6722bf0ab394e588ced846a280a8096 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:55.446762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:55.448426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:55.462347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71f2c8-3a43-4d40-90c1-8f556177df9d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:55.521323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:55.522192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:55.522474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:55.522747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] PATH_INFO: `/v3/projects/0fc6d31207224e41a2e6c2e95e09af03` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:55.523924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc6d31207224e41a2e6c2e95e09af03)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:55.525864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:55.580775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc6d31207224e41a2e6c2e95e09af03', 'request_id': 'req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.591675 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc6d31207224e41a2e6c2e95e09af03', 'request_id': 'req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.600207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc6d31207224e41a2e6c2e95e09af03', 'request_id': 'req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.611260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] The token cache is being invalidate because project 0fc6d31207224e41a2e6c2e95e09af03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:19:55.611443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.612139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1a68ea6a-e44f-4a62-9c5b-d2d266ae5074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:19:55.614126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1233/2482] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:19:55 2022] DELETE /identity/v3/projects/0fc6d31207224e41a2e6c2e95e09af03 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:55.619261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14409728-c136-4839-a0a9-6f4bd121139b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:55.619553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14409728-c136-4839-a0a9-6f4bd121139b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:55.619831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14409728-c136-4839-a0a9-6f4bd121139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:55.642612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-14409728-c136-4839-a0a9-6f4bd121139b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:55.643823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-14409728-c136-4839-a0a9-6f4bd121139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:55.707221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1250/2483] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:55.715399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7c21-fac0-40f0-ab2c-a3270be42229 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:55.715684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7c21-fac0-40f0-ab2c-a3270be42229 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:55.715958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7c21-fac0-40f0-ab2c-a3270be42229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:55.862889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-603a7c21-fac0-40f0-ab2c-a3270be42229 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:55.864042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-603a7c21-fac0-40f0-ab2c-a3270be42229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:55.872834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1234/2484] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:55.877210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14409728-c136-4839-a0a9-6f4bd121139b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:55.888964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14409728-c136-4839-a0a9-6f4bd121139b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:55.894300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:55.895107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:55.895506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:55.895794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:55.896519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:19:55.896823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:55.898487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:55.913104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1251/2485] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:19:55.917673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a7c21-fac0-40f0-ab2c-a3270be42229 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:55.927075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a7c21-fac0-40f0-ab2c-a3270be42229 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:55.931978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:55.932632 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:55.932909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:55.933217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:55.933902 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:55.935787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:55.962828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1235/2486] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:55 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:55.967312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:55.978780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a63295-ce1e-44d4-9bcf-f7cdd2716ba4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.051292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.052933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.053411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.053669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.054454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.056449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.085033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1252/2487] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:55 2022] POST /identity/v3/users => generated 305 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:56.089745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.101695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc308-ee28-408a-867d-e515a318054f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.106464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.107135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.107418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.107693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.108430 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.110059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.116243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:56.126339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.127290 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1236/2488] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:19:56.136976 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eeb20-11de-4e19-9ac5-54888aa66e03 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.142051 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39235830-f4b3-459b-bd92-6bcf325d9420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.142712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39235830-f4b3-459b-bd92-6bcf325d9420 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.143026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39235830-f4b3-459b-bd92-6bcf325d9420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.143303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39235830-f4b3-459b-bd92-6bcf325d9420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.144042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39235830-f4b3-459b-bd92-6bcf325d9420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.145888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39235830-f4b3-459b-bd92-6bcf325d9420 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.156406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1253/2489] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:56.160943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.171746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34dd17-62e2-413e-adcf-a190fecfd1bf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.177036 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.177917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.178195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.178461 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] PATH_INFO: `/v3/projects/6c0616cc4ae94fcbb4797b717ab9b6f6/users/18b4cd5dcfaa452fb63d7230783ae072/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.193624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0616cc4ae94fcbb4797b717ab9b6f6, user_id=18b4cd5dcfaa452fb63d7230783ae072, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.196685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.212179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1237/2490] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:56 2022] PUT /identity/v3/projects/6c0616cc4ae94fcbb4797b717ab9b6f6/users/18b4cd5dcfaa452fb63d7230783ae072/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:19:56.279634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8b1d-bf04-43d7-a805-5bc4e1600400 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.279781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8b1d-bf04-43d7-a805-5bc4e1600400 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.280414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8b1d-bf04-43d7-a805-5bc4e1600400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.308239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cc4c8b1d-bf04-43d7-a805-5bc4e1600400 None None] MFA Rules not processed for user `18b4cd5dcfaa452fb63d7230783ae072`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:56.309520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cc4c8b1d-bf04-43d7-a805-5bc4e1600400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:56.322997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:56.378138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.380273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:56.380456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1254/2491] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:56.385084 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c8b1d-bf04-43d7-a805-5bc4e1600400 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.438469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91eb98-2526-4a2f-bd05-af37c25f658b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.442059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.442798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.443083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.443353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.445154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:56.449002 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c8b1d-bf04-43d7-a805-5bc4e1600400 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.453775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.454438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.454714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.454986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.455668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.457626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.487362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1255/2492] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:19:56 2022] POST /identity/v3/projects => generated 409 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:19:56.488352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.505891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.507327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.516608 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1238/2493] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:56.549868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1506d-185b-4a27-99a5-f43e1c96f5c9 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.556565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.557269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.557557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.557829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.558520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.560424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.586349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1256/2494] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:19:56 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:19:56.590684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.602201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9c173-d744-4b33-a9d1-374253894f1f service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.607053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.607810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.608143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.608502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.609372 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.611034 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.621883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1239/2495] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:56.626055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.639202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d918c-bdff-4af2-8be2-947e38c06551 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.644932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.646025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.646268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.646588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] PATH_INFO: `/v3/projects/106ccf6ab8844359a7ac0fb14bef4d63/users/7a0349b38c784f7286f3dfb27106a3b3/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.664963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=106ccf6ab8844359a7ac0fb14bef4d63, user_id=7a0349b38c784f7286f3dfb27106a3b3, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.667623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.686070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1257/2496] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:56 2022] PUT /identity/v3/projects/106ccf6ab8844359a7ac0fb14bef4d63/users/7a0349b38c784f7286f3dfb27106a3b3/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:56.690449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.700919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522fcd1-85e5-4d67-b245-65978ea7e6c7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.759613 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc1aee-069b-4410-a9d7-b1a08420df49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.760321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc1aee-069b-4410-a9d7-b1a08420df49 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.760594 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc1aee-069b-4410-a9d7-b1a08420df49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.760868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc1aee-069b-4410-a9d7-b1a08420df49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.761614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc1aee-069b-4410-a9d7-b1a08420df49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.763194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc1aee-069b-4410-a9d7-b1a08420df49 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.773068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1240/2497] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:19:56.777359 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.784622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:56.789350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee549a0-d888-4d2a-9908-ca9541ebb127 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:56.795605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:56.796795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.797363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.797363 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] PATH_INFO: `/v3/projects/106ccf6ab8844359a7ac0fb14bef4d63/users/7a0349b38c784f7286f3dfb27106a3b3/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.799807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=106ccf6ab8844359a7ac0fb14bef4d63, user_id=7a0349b38c784f7286f3dfb27106a3b3, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:56.802864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:56.814533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1258/2498] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:19:56 2022] PUT /identity/v3/projects/106ccf6ab8844359a7ac0fb14bef4d63/users/7a0349b38c784f7286f3dfb27106a3b3/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:19:56.825024 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3eb86-2ff8-4f7d-a887-328d5b5d6473 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:56.825332 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3eb86-2ff8-4f7d-a887-328d5b5d6473 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:56.825605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3eb86-2ff8-4f7d-a887-328d5b5d6473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:56.853836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8cb3eb86-2ff8-4f7d-a887-328d5b5d6473 None None] MFA Rules not processed for user `7a0349b38c784f7286f3dfb27106a3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:19:56.855101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8cb3eb86-2ff8-4f7d-a887-328d5b5d6473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:56.860007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:19:56.934834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1241/2499] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:19:56.945693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:56.947801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:57.009149 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03825d38-1807-4561-b70c-d780a1b83c8d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:57.013272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:57.014484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:57.014797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:57.015068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:57.025190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:57.026646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:57.037079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1259/2500] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:57.121768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3eb86-2ff8-4f7d-a887-328d5b5d6473 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:57.133288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3eb86-2ff8-4f7d-a887-328d5b5d6473 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:57.137042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:57.137872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:57.138159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:57.138503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:57.141513 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:19:57.210826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:57.212268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:57.221951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1242/2501] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:19:57.238482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:19:57.248390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e3b5-a575-420c-9c28-b5bc5408bc0c service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:19:57.252812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf0657-6247-4307-ad17-8d8a266817ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:19:57.253490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf0657-6247-4307-ad17-8d8a266817ea service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:19:57.253660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf0657-6247-4307-ad17-8d8a266817ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:19:57.253971 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf0657-6247-4307-ad17-8d8a266817ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:19:57.323472 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf0657-6247-4307-ad17-8d8a266817ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:19:57.325315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf0657-6247-4307-ad17-8d8a266817ea service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:19:57.339429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1260/2502] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:19:57 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:03.244061 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:03.246136 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:03.326940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b44f-8e9a-4aa0-bd3a-f23ab4f99f50 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:03.331657 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:03.332547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:03.333307 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:03.333589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] PATH_INFO: `/v3/users/2eef71100dcc45caa7f13d362777eb38` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:03.335827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eef71100dcc45caa7f13d362777eb38)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:03.338537 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:03.406503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:03.417236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eef71100dcc45caa7f13d362777eb38'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:03.417898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eef71100dcc45caa7f13d362777eb38'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:03.436605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eef71100dcc45caa7f13d362777eb38'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:03.437409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eef71100dcc45caa7f13d362777eb38'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:03.446269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eef71100dcc45caa7f13d362777eb38'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:03.462159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1243/2503] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:03 2022] DELETE /identity/v3/users/2eef71100dcc45caa7f13d362777eb38 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:03.466783 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:03.467303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:03.467962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:03.485689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:03.486894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:03.549859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1261/2504] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:03.560221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:03.561983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:03.630163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c2f71-6990-4f1e-96b8-4eac7ea8fdf2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:03.634231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:03.634963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:03.635192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:03.635451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:03.644857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:03.646368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:03.656889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1244/2505] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:03.995931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:03.997390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:04.009655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252849d-f38f-41e5-8a38-feb295bb15ea None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:04.014220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:04.015099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:04.015388 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:04.015652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] PATH_INFO: `/v3/projects/b57b978a818a4f1bb7caaab37bb46f45` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:04.016832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57b978a818a4f1bb7caaab37bb46f45)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:04.018816 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:04.051521 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:04.068151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57b978a818a4f1bb7caaab37bb46f45', 'request_id': 'req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.075383 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57b978a818a4f1bb7caaab37bb46f45', 'request_id': 'req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.080959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57b978a818a4f1bb7caaab37bb46f45', 'request_id': 'req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.094625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] The token cache is being invalidate because project b57b978a818a4f1bb7caaab37bb46f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:04.094947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.095548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.097175 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1262/2506] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:03 2022] DELETE /identity/v3/projects/b57b978a818a4f1bb7caaab37bb46f45 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:04.102936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:04.112518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3862-1109-4bb0-a8af-81da788ab1f3 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:04.174642 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:04.175519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:04.175803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:04.176068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] PATH_INFO: `/v3/users/f3dddc2411da4cdfaf855f694105f766` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:04.177353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dddc2411da4cdfaf855f694105f766)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:04.179406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:04.233252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dddc2411da4cdfaf855f694105f766'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.233926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dddc2411da4cdfaf855f694105f766'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.241842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:04.246988 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dddc2411da4cdfaf855f694105f766'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.247723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dddc2411da4cdfaf855f694105f766'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.255615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dddc2411da4cdfaf855f694105f766'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.268305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1245/2507] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:04 2022] DELETE /identity/v3/users/f3dddc2411da4cdfaf855f694105f766 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:04.548933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:04.550554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:04.564753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2c04-9a4c-4a35-9fc2-2db5ca76fbe1 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:04.632760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:04.633680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:04.633960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:04.634228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] PATH_INFO: `/v3/projects/0187c8f8308c42a6b3ea2566c9c32d9e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:04.635422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0187c8f8308c42a6b3ea2566c9c32d9e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:04.637405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:04.685361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0187c8f8308c42a6b3ea2566c9c32d9e', 'request_id': 'req-e84efc97-29f6-4e20-8b11-68dfad90a83a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.691565 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0187c8f8308c42a6b3ea2566c9c32d9e', 'request_id': 'req-e84efc97-29f6-4e20-8b11-68dfad90a83a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.697009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0187c8f8308c42a6b3ea2566c9c32d9e', 'request_id': 'req-e84efc97-29f6-4e20-8b11-68dfad90a83a'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.709642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] The token cache is being invalidate because project 0187c8f8308c42a6b3ea2566c9c32d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:04.709926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.710546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.712094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1263/2508] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:04 2022] DELETE /identity/v3/projects/0187c8f8308c42a6b3ea2566c9c32d9e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:04.716209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:04.717623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:04.739511 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f9a3-51bd-4847-9222-dead843e55fe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:04.800787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:04.801615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:04.801965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:04.802476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] PATH_INFO: `/v3/users/63a117dbb0f449df92d39125726c1d87` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:04.803780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a117dbb0f449df92d39125726c1d87)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:04.805827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:04.858730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a117dbb0f449df92d39125726c1d87'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.859398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a117dbb0f449df92d39125726c1d87'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.872552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a117dbb0f449df92d39125726c1d87'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.873378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a117dbb0f449df92d39125726c1d87'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.882719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36dd74c1-26e4-443c-9cb4-c656e364a1f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a117dbb0f449df92d39125726c1d87'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:04.896904 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1246/2509] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:04 2022] DELETE /identity/v3/users/63a117dbb0f449df92d39125726c1d87 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:05.326964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.328601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:05.341376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84efc97-29f6-4e20-8b11-68dfad90a83a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.397704 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:05.414910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.415796 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.416077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.416347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] PATH_INFO: `/v3/projects/75d73d45e3524dc6945d6fecfcfac516` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.417776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75d73d45e3524dc6945d6fecfcfac516)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.419725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.471152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75d73d45e3524dc6945d6fecfcfac516', 'request_id': 'req-d9a1796d-445f-4687-b79e-71856543abb1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.479356 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75d73d45e3524dc6945d6fecfcfac516', 'request_id': 'req-d9a1796d-445f-4687-b79e-71856543abb1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.486010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75d73d45e3524dc6945d6fecfcfac516', 'request_id': 'req-d9a1796d-445f-4687-b79e-71856543abb1'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.498858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] The token cache is being invalidate because project 75d73d45e3524dc6945d6fecfcfac516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:05.499150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.499784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9a1796d-445f-4687-b79e-71856543abb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.505184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1264/2510] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:05 2022] DELETE /identity/v3/projects/75d73d45e3524dc6945d6fecfcfac516 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:05.506886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813bdd3-a30b-4f7e-a8ee-f42337e689f1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.507343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813bdd3-a30b-4f7e-a8ee-f42337e689f1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.507440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813bdd3-a30b-4f7e-a8ee-f42337e689f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.525851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4813bdd3-a30b-4f7e-a8ee-f42337e689f1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:05.527078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4813bdd3-a30b-4f7e-a8ee-f42337e689f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:05.575694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:05.591392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1247/2511] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:05.599365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156887-f42d-40b0-a3b0-d5d73cb16cc7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.599920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156887-f42d-40b0-a3b0-d5d73cb16cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.600440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156887-f42d-40b0-a3b0-d5d73cb16cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.618765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8c156887-f42d-40b0-a3b0-d5d73cb16cc7 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:05.620017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8c156887-f42d-40b0-a3b0-d5d73cb16cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:05.628258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1265/2512] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:05.632567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813bdd3-a30b-4f7e-a8ee-f42337e689f1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.642928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813bdd3-a30b-4f7e-a8ee-f42337e689f1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.647880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.648525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.648812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.649190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.649952 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:05.650262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.652235 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.665008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1248/2513] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:05.670658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c156887-f42d-40b0-a3b0-d5d73cb16cc7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.679974 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c156887-f42d-40b0-a3b0-d5d73cb16cc7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.685554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.686294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.686644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.686986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.687800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.689829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.717096 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1266/2514] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:05 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:05.722080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.734497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cf1a0-b974-4476-a2e0-6bb30f896d65 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.776834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.778776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:05.793976 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.794649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.794958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.795233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.795918 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.797874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.821401 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1249/2515] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:05 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:05.822501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.832281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76d7d-f510-49a9-8c8c-508a6db12298 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.836772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.837455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.837738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.838013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.838842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.840447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.850029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea2f62-2493-4ccd-b357-88e28a7113ed admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.855318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1250/2516] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:20:05.855485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.856352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.856650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.856956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] PATH_INFO: `/v3/users/18b4cd5dcfaa452fb63d7230783ae072` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.857101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.858358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b4cd5dcfaa452fb63d7230783ae072)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.860279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.867617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4fc21-b38c-48cd-aba7-90ede56fdf1a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.872390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.873133 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.873418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.873693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.874420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.876001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.886345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1251/2517] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:20:05.891004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:05.903927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6326a36-0a73-41bf-b382-64d6b895d36c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:05.911228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc5ccd-7055-4944-9ffe-7c41ad745dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:05.912656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc5ccd-7055-4944-9ffe-7c41ad745dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.913159 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc5ccd-7055-4944-9ffe-7c41ad745dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.913618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc5ccd-7055-4944-9ffe-7c41ad745dfa admin admin] PATH_INFO: `/v3/projects/f15d014b88a444c28ae32f230d144e8c/users/7e060034139f420e8412b8a55f1b29a2/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.917824 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b4cd5dcfaa452fb63d7230783ae072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.918611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b4cd5dcfaa452fb63d7230783ae072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.919493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b4cd5dcfaa452fb63d7230783ae072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.921629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:05.932456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc5ccd-7055-4944-9ffe-7c41ad745dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15d014b88a444c28ae32f230d144e8c, user_id=7e060034139f420e8412b8a55f1b29a2, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:05.934904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b4cd5dcfaa452fb63d7230783ae072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.935668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc5ccd-7055-4944-9ffe-7c41ad745dfa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:05.942405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9d462c31-9bf6-49cb-8bbb-4151e1ada4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b4cd5dcfaa452fb63d7230783ae072'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:05.951907 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1252/2518] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:05 2022] PUT /identity/v3/projects/f15d014b88a444c28ae32f230d144e8c/users/7e060034139f420e8412b8a55f1b29a2/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:05.957482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e717c-049a-4405-a660-82c142791e17 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.957832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e717c-049a-4405-a660-82c142791e17 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.958111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e717c-049a-4405-a660-82c142791e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.960370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1267/2519] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:05 2022] DELETE /identity/v3/users/18b4cd5dcfaa452fb63d7230783ae072 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:05.964844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:05.965162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:05.965438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:05.976391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d77e717c-049a-4405-a660-82c142791e17 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:05.977625 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d77e717c-049a-4405-a660-82c142791e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:05.983955 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:05.985224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:06.049894 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1253/2520] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:06.054379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1268/2521] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:06.059990 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e717c-049a-4405-a660-82c142791e17 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:06.061438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d77e717c-049a-4405-a660-82c142791e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:06.066352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:06.068859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:06.094166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:06.127867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e717c-049a-4405-a660-82c142791e17 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:06.132327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:06.132554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:06.132822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:06.133125 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:06.136887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b6bad-5e7c-4c6e-b0ab-a75e77aa9846 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:06.140898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:06.141606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:06.141890 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:06.142333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:06.143089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:06.144652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:06.153082 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:06.154930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:06.155086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1254/2522] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:06.169958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1269/2523] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:20:06.502405 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:06.503791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:06.523937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdb0cf-0901-4e76-94e7-3555c248db64 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:06.523937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:06.524611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:06.524611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:06.524801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] PATH_INFO: `/v3/projects/6c0616cc4ae94fcbb4797b717ab9b6f6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:06.526993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0616cc4ae94fcbb4797b717ab9b6f6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:06.529213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:06.579881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0616cc4ae94fcbb4797b717ab9b6f6', 'request_id': 'req-c8678bdd-a594-4a11-ba42-c57b563953c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.591679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0616cc4ae94fcbb4797b717ab9b6f6', 'request_id': 'req-c8678bdd-a594-4a11-ba42-c57b563953c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.599644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0616cc4ae94fcbb4797b717ab9b6f6', 'request_id': 'req-c8678bdd-a594-4a11-ba42-c57b563953c8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.612343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] The token cache is being invalidate because project 6c0616cc4ae94fcbb4797b717ab9b6f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:06.612702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.613622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c8678bdd-a594-4a11-ba42-c57b563953c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.615378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1255/2524] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:06 2022] DELETE /identity/v3/projects/6c0616cc4ae94fcbb4797b717ab9b6f6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:06.619868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:06.630199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072fc48-a82f-4307-b176-d3413c45e3b7 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:06.692269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:06.693554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:06.693554 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:06.693762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] PATH_INFO: `/v3/users/7a0349b38c784f7286f3dfb27106a3b3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:06.695143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0349b38c784f7286f3dfb27106a3b3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:06.697121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:06.711168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:06.747657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0349b38c784f7286f3dfb27106a3b3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.748379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0349b38c784f7286f3dfb27106a3b3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.749027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0349b38c784f7286f3dfb27106a3b3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.762837 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0349b38c784f7286f3dfb27106a3b3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.771187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0349b38c784f7286f3dfb27106a3b3'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:06.784771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1270/2525] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:06 2022] DELETE /identity/v3/users/7a0349b38c784f7286f3dfb27106a3b3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:06.907139 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:06.907425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:06.908588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:06.942924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] MFA Rules not processed for user `7e060034139f420e8412b8a55f1b29a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:06.944484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:07.039007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1256/2526] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:07.051109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.052642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:07.114780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aaf97-1a4e-404a-9169-19c92932803d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.118860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57624e4-6f36-411a-863e-0275304d09aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.119780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57624e4-6f36-411a-863e-0275304d09aa service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.120262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57624e4-6f36-411a-863e-0275304d09aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.120764 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57624e4-6f36-411a-863e-0275304d09aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.131888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57624e4-6f36-411a-863e-0275304d09aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.133384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57624e4-6f36-411a-863e-0275304d09aa service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.142923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1271/2527] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:20:07.319829 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.321462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:07.384798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:07.428897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5a59-1484-4d36-8dee-45ca42b132bd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.435020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.435878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.436319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.436604 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] PATH_INFO: `/v3/projects/106ccf6ab8844359a7ac0fb14bef4d63` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.437850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] RBAC: Authorizing `identity:delete_project(project_id=106ccf6ab8844359a7ac0fb14bef4d63)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.440362 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.496834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106ccf6ab8844359a7ac0fb14bef4d63', 'request_id': 'req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:07.507949 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106ccf6ab8844359a7ac0fb14bef4d63', 'request_id': 'req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:07.515828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106ccf6ab8844359a7ac0fb14bef4d63', 'request_id': 'req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:07.533267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] The token cache is being invalidate because project 106ccf6ab8844359a7ac0fb14bef4d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:07.533835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:07.534319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:07.536170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1257/2528] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:07 2022] DELETE /identity/v3/projects/106ccf6ab8844359a7ac0fb14bef4d63 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:07.541683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a331b-7fa9-4588-a0b1-0a6ad13596a6 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.542237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a331b-7fa9-4588-a0b1-0a6ad13596a6 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.543292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a331b-7fa9-4588-a0b1-0a6ad13596a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.574023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-fc7a331b-7fa9-4588-a0b1-0a6ad13596a6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:07.575575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fc7a331b-7fa9-4588-a0b1-0a6ad13596a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:07.636208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.637318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:07.650843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1272/2529] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:07.658956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061a29-9804-493d-80ee-25e1664a20a7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.659249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061a29-9804-493d-80ee-25e1664a20a7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.659655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72061a29-9804-493d-80ee-25e1664a20a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.683385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-72061a29-9804-493d-80ee-25e1664a20a7 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:07.684675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-72061a29-9804-493d-80ee-25e1664a20a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:07.692647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1273/2530] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:07.696749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72061a29-9804-493d-80ee-25e1664a20a7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.700871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019310c-f90e-4ab7-b8b8-4f4fcf3c899c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.705189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.705832 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.706119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.706223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72061a29-9804-493d-80ee-25e1664a20a7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.706381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.708155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:07.710954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.711595 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.711894 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.712151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.712878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:07.713213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.715047 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.721546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.723048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.727102 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1274/2531] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:20:07.731455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.734472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1258/2532] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 07:20:07.748665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5747c8f-e98a-442a-b458-b509ce061d78 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.753787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.754525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.754875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.755215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.756001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.758004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.768895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:07.791669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1275/2533] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:07 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:20:07.795635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.805910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7247c-c740-45d9-bf4d-2a9bb88670d9 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.864469 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.865180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.865467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.865744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.866440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.868361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.893411 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1259/2534] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:07 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:07.894951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.907196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc8fc0-eb47-497d-a6a4-78e48105f9da admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.912417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.913129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.913417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.913693 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.914448 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.916104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.927698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1276/2535] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:07.931925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.939320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:07.944636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57a19f-20bb-48da-8909-5f95d51eb501 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.951294 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a2ba1-ed8d-444e-8c61-37c5118240e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.952102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2ba1-ed8d-444e-8c61-37c5118240e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.952393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2ba1-ed8d-444e-8c61-37c5118240e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.952678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2ba1-ed8d-444e-8c61-37c5118240e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:07.953823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a2ba1-ed8d-444e-8c61-37c5118240e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:07.956067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a2ba1-ed8d-444e-8c61-37c5118240e5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:07.966981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1260/2536] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:20:07.971323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:07.983352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2527f8-a37b-488f-a04a-33d21425e073 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:07.994109 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:07.995537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:07.996467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:07.996467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] PATH_INFO: `/v3/projects/00662d58033946ffb970e3892aae9e3f/users/366e0d94a93c43958349e333a1edd060/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:08.013454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=00662d58033946ffb970e3892aae9e3f, user_id=366e0d94a93c43958349e333a1edd060, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:08.016125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:08.031557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1277/2537] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:07 2022] PUT /identity/v3/projects/00662d58033946ffb970e3892aae9e3f/users/366e0d94a93c43958349e333a1edd060/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:08.042478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02b25a-3935-46fb-8969-d75f077d16bf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:08.042800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02b25a-3935-46fb-8969-d75f077d16bf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:08.043217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02b25a-3935-46fb-8969-d75f077d16bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:08.072173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0b02b25a-3935-46fb-8969-d75f077d16bf None None] MFA Rules not processed for user `366e0d94a93c43958349e333a1edd060`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:08.073439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0b02b25a-3935-46fb-8969-d75f077d16bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:08.142529 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1261/2538] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:08.154874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:08.156709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:08.217484 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea55f3b-3813-45c8-994d-b4995a387dae admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:08.221422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366a7e7596d14b32a6ce62790a2e3dd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:08.222174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:08.222456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:08.222724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:08.232128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:08.233634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:08.243121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1278/2539] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:08.559182 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d30365-ca29-420c-aa28-5963e4d292a0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:08.560759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d30365-ca29-420c-aa28-5963e4d292a0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:08.561379 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d30365-ca29-420c-aa28-5963e4d292a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:08.583179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f0d30365-ca29-420c-aa28-5963e4d292a0 None None] MFA Rules not processed for user `7e060034139f420e8412b8a55f1b29a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:08.584811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f0d30365-ca29-420c-aa28-5963e4d292a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:08.656970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1262/2540] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:11.764747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:11.766294 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:11.855314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e417f7a-2e3c-4a78-b02d-617e535a3b8d service glance] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:11.860733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:11.861606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:11.861776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:11.862058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] PATH_INFO: `/v3/users/ef6395fa14b54c43931ffeaab023d97c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:11.863327 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6395fa14b54c43931ffeaab023d97c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:11.865288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:11.927872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6395fa14b54c43931ffeaab023d97c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:11.928589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6395fa14b54c43931ffeaab023d97c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:11.929278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6395fa14b54c43931ffeaab023d97c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:11.939789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:11.942223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6395fa14b54c43931ffeaab023d97c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:11.950657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-5c7cd6ba-9917-449e-b966-a6ce936ccbdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6395fa14b54c43931ffeaab023d97c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:11.964077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1279/2541] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:11 2022] DELETE /identity/v3/users/ef6395fa14b54c43931ffeaab023d97c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:12.162876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d30365-ca29-420c-aa28-5963e4d292a0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.177764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d30365-ca29-420c-aa28-5963e4d292a0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:12.244780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:12.246730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.247020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.247297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] PATH_INFO: `/v3/projects/b52e5f4c43a3469db851b4704f422448` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.249850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52e5f4c43a3469db851b4704f422448)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:12.251786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:12.287607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:12.301841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52e5f4c43a3469db851b4704f422448', 'request_id': 'req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:12.309876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52e5f4c43a3469db851b4704f422448', 'request_id': 'req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:12.316398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52e5f4c43a3469db851b4704f422448', 'request_id': 'req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:12.327420 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] The token cache is being invalidate because project b52e5f4c43a3469db851b4704f422448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:12.327702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:12.328336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-37ed7b0d-4a5d-4bb5-8af8-c6bf9541ccd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:12.330343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1263/2542] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:12 2022] DELETE /identity/v3/projects/b52e5f4c43a3469db851b4704f422448 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:12.335334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09469be-c8c0-4803-810c-10f621713062 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.335623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09469be-c8c0-4803-810c-10f621713062 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.335914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09469be-c8c0-4803-810c-10f621713062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.354826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b09469be-c8c0-4803-810c-10f621713062 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:12.356033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b09469be-c8c0-4803-810c-10f621713062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:12.417924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1280/2543] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:12.426072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3e68-3d0b-4c70-a762-65b1401b91c6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.426364 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3e68-3d0b-4c70-a762-65b1401b91c6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.426807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3e68-3d0b-4c70-a762-65b1401b91c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.447931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d4fe3e68-3d0b-4c70-a762-65b1401b91c6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:12.449316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d4fe3e68-3d0b-4c70-a762-65b1401b91c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:12.458009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1264/2544] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:12.462314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09469be-c8c0-4803-810c-10f621713062 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.472332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09469be-c8c0-4803-810c-10f621713062 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:12.477004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:12.477735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.478013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.478287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.479184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:12.479486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:12.481126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:12.493183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1281/2545] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:12.497458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3e68-3d0b-4c70-a762-65b1401b91c6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.506775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3e68-3d0b-4c70-a762-65b1401b91c6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:12.511821 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:12.512478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.512758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.513026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.513740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:12.515626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:12.545403 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1265/2546] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:12 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:12.549612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.560032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11900be7-5aa0-4d1c-985a-5d5eb4c67b0f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:12.616365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:12.617044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.617353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.617623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.618337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:12.620628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:12.643756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1282/2547] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:12 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:12.647318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.658656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018fd48-bdf6-4e8e-926a-46031196a7db admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:12.665277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:12.665961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.666516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.666643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.667384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:12.669106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:12.680269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1266/2548] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:12.684449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.694742 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c5ede-2899-4f59-91c4-bbaafa6095eb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:12.699524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abddb46-8705-4af5-8dc3-46845b181358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:12.700199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abddb46-8705-4af5-8dc3-46845b181358 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.700487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abddb46-8705-4af5-8dc3-46845b181358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.700790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abddb46-8705-4af5-8dc3-46845b181358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.701692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abddb46-8705-4af5-8dc3-46845b181358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:12.703319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abddb46-8705-4af5-8dc3-46845b181358 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:12.713910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1283/2549] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:12.718344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.728622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07e598-8ce1-441c-844d-ba17ac047d4c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:12.733646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:12.734600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.734883 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.735161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] PATH_INFO: `/v3/projects/8fb3fd6cb2f643d9adb56409a6b98341/users/f056820fb03e45eda2b3b76cad60ed95/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.750228 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb3fd6cb2f643d9adb56409a6b98341, user_id=f056820fb03e45eda2b3b76cad60ed95, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:12.752914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:12.767350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1267/2550] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:12 2022] PUT /identity/v3/projects/8fb3fd6cb2f643d9adb56409a6b98341/users/f056820fb03e45eda2b3b76cad60ed95/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:12.778617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec196a-f2b0-4624-afbe-bd4360493911 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:12.779150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec196a-f2b0-4624-afbe-bd4360493911 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:12.779150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec196a-f2b0-4624-afbe-bd4360493911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:12.808215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-12ec196a-f2b0-4624-afbe-bd4360493911 None None] MFA Rules not processed for user `f056820fb03e45eda2b3b76cad60ed95`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:12.809652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-12ec196a-f2b0-4624-afbe-bd4360493911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:12.850986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:12.898753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1284/2551] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:12.926243 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:12.933887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:13.027218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142df881-0b26-408c-924f-ef100609ef98 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:13.036144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39864ed2-31e0-481e-833c-24d545b76f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:13.036833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39864ed2-31e0-481e-833c-24d545b76f4b service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:13.037155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39864ed2-31e0-481e-833c-24d545b76f4b service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:13.037453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39864ed2-31e0-481e-833c-24d545b76f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:13.046486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39864ed2-31e0-481e-833c-24d545b76f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:13.048097 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39864ed2-31e0-481e-833c-24d545b76f4b service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:13.057331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1268/2552] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:13.298256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec196a-f2b0-4624-afbe-bd4360493911 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:13.360601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec196a-f2b0-4624-afbe-bd4360493911 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:13.371106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:13.372088 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:13.372353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:13.372634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] PATH_INFO: `/v3/users/f056820fb03e45eda2b3b76cad60ed95` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:13.373926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f056820fb03e45eda2b3b76cad60ed95)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:13.376021 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:13.430533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f056820fb03e45eda2b3b76cad60ed95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:13.431151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f056820fb03e45eda2b3b76cad60ed95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:13.431746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f056820fb03e45eda2b3b76cad60ed95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:13.444036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f056820fb03e45eda2b3b76cad60ed95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:13.451362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f056820fb03e45eda2b3b76cad60ed95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:13.465471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1285/2553] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:13 2022] DELETE /identity/v3/users/f056820fb03e45eda2b3b76cad60ed95 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:13.470170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:13.470690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:13.470822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:13.493607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:13.495100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:13.528730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:13.556703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1269/2554] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:13.566799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:13.568455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:13.758453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d849c49-7470-4850-aef3-d992245b48cd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:13.762422 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7defd-08a5-40d1-99e6-8fb414335fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:13.763078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7defd-08a5-40d1-99e6-8fb414335fde service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:13.763357 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7defd-08a5-40d1-99e6-8fb414335fde service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:13.763633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7defd-08a5-40d1-99e6-8fb414335fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:13.770849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:13.773279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7defd-08a5-40d1-99e6-8fb414335fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:13.775026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7defd-08a5-40d1-99e6-8fb414335fde service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:13.785710 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1286/2555] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:14.070819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.072496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:14.087477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9334c7f-1659-42f1-b16d-7ae72380af91 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.093498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.094390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.094679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.094946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] PATH_INFO: `/v3/projects/8fb3fd6cb2f643d9adb56409a6b98341` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.096186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb3fd6cb2f643d9adb56409a6b98341)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.098166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.151638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb3fd6cb2f643d9adb56409a6b98341', 'request_id': 'req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:14.161033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb3fd6cb2f643d9adb56409a6b98341', 'request_id': 'req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:14.176562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb3fd6cb2f643d9adb56409a6b98341', 'request_id': 'req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:14.188071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] The token cache is being invalidate because project 8fb3fd6cb2f643d9adb56409a6b98341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:14.188474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:14.189168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b3678fd6-725e-44d2-baa7-0ec3baddd4c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:14.190742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1270/2556] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:14 2022] DELETE /identity/v3/projects/8fb3fd6cb2f643d9adb56409a6b98341 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:14.195619 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fac7c2-2988-41f4-8b2d-2e0e186f016b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.195956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fac7c2-2988-41f4-8b2d-2e0e186f016b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.196232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fac7c2-2988-41f4-8b2d-2e0e186f016b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.214829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b0fac7c2-2988-41f4-8b2d-2e0e186f016b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:14.216126 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b0fac7c2-2988-41f4-8b2d-2e0e186f016b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:14.283703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1287/2557] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:14.285399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431064b8-3bb3-4cbf-8f2c-52434a195cd0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.285688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431064b8-3bb3-4cbf-8f2c-52434a195cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.285965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431064b8-3bb3-4cbf-8f2c-52434a195cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.304323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-431064b8-3bb3-4cbf-8f2c-52434a195cd0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:14.305570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-431064b8-3bb3-4cbf-8f2c-52434a195cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:14.314320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1271/2558] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:14.319261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fac7c2-2988-41f4-8b2d-2e0e186f016b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.335658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fac7c2-2988-41f4-8b2d-2e0e186f016b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.342271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.342937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.343221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.343497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.344330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:14.344638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.346325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.361107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1288/2559] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:14.364973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431064b8-3bb3-4cbf-8f2c-52434a195cd0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.375106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431064b8-3bb3-4cbf-8f2c-52434a195cd0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.379777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.380508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.380789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.381087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.381801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.383707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.411720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1272/2560] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:14 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:14.416321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.426293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fb395-ab4d-4e0b-8520-612cd1ea0ddb admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.487885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.488662 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.488949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.489257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.489960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.491881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.515355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1289/2561] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:14 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:14.522414 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.534453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5533ea1-dcdb-4c03-a28e-4a8fefa0a600 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.539979 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.540649 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.540928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.541689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.542463 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.544118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.556864 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1273/2562] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:14.561362 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.572108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e80770-1ff1-4a2e-a6ec-fda7345b0245 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.577024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b77ba8-4477-4a48-bf0c-58a4492c4e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.577767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b77ba8-4477-4a48-bf0c-58a4492c4e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.578048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b77ba8-4477-4a48-bf0c-58a4492c4e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.578482 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b77ba8-4477-4a48-bf0c-58a4492c4e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.579235 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b77ba8-4477-4a48-bf0c-58a4492c4e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.580854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b77ba8-4477-4a48-bf0c-58a4492c4e2b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.595379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1290/2563] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:14.596698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.612515 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea2235-7fa8-40a0-99fb-5ccefcdf66ec admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.618486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.619470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.619919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.620314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] PATH_INFO: `/v3/projects/454ce4fb8c9145e983b8fa96b0b43584/users/95fdb9c3caea4faab18d209dbb077cdc/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.637074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454ce4fb8c9145e983b8fa96b0b43584, user_id=95fdb9c3caea4faab18d209dbb077cdc, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.639772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.660164 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1274/2564] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:14 2022] PUT /identity/v3/projects/454ce4fb8c9145e983b8fa96b0b43584/users/95fdb9c3caea4faab18d209dbb077cdc/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:14.672996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afae3ab-110c-4e5e-ac93-3cc0dcf02d24 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.673498 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afae3ab-110c-4e5e-ac93-3cc0dcf02d24 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.673981 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afae3ab-110c-4e5e-ac93-3cc0dcf02d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.705042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9afae3ab-110c-4e5e-ac93-3cc0dcf02d24 None None] MFA Rules not processed for user `95fdb9c3caea4faab18d209dbb077cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:14.706284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9afae3ab-110c-4e5e-ac93-3cc0dcf02d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:14.757930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:14.783698 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1291/2565] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:14.788209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.860051 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2afae8-a4c6-44c3-b908-58051826bd13 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.867892 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.868593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.868877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.869183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.870086 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.872093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:14.876074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:14.908196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1275/2566] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:14 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:20:14.912635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afae3ab-110c-4e5e-ac93-3cc0dcf02d24 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:14.924066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afae3ab-110c-4e5e-ac93-3cc0dcf02d24 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:14.982839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:14.983871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:14.984153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:14.984437 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:14.985152 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:14.991582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:15.008613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1292/2567] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:14 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:15.013335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:15.023938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2e5b-a216-401b-a418-8bf0a05e9c3c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:15.028690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:15.029771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:15.030046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:15.030316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:15.031042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:15.032629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:15.046689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1276/2568] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:15.047850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:15.058464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b9a77-ef68-4533-b67c-43759cd25f39 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:15.064954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:15.065989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:15.066344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:15.066724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] PATH_INFO: `/v3/projects/f801adde056c45a0b05b46ed8ca76d47/users/637a011a165c4759a5bcd9df74880e17/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:15.089450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f801adde056c45a0b05b46ed8ca76d47, user_id=637a011a165c4759a5bcd9df74880e17, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:15.095524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:15.112630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1293/2569] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:15 2022] PUT /identity/v3/projects/f801adde056c45a0b05b46ed8ca76d47/users/637a011a165c4759a5bcd9df74880e17/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:15.116807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:15.127105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa197a34-6672-4245-9167-bd413693095f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:15.186260 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef6687-c71e-4f3a-b266-ff55994f6316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:15.186939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef6687-c71e-4f3a-b266-ff55994f6316 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:15.187222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef6687-c71e-4f3a-b266-ff55994f6316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:15.187496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef6687-c71e-4f3a-b266-ff55994f6316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:15.188229 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef6687-c71e-4f3a-b266-ff55994f6316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:15.189972 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef6687-c71e-4f3a-b266-ff55994f6316 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:15.200744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1277/2570] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:15.205731 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:15.216991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc939d5-bc8a-41f3-859b-6716fd54f9fa admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:15.223032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:15.223937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:15.224224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:15.224499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] PATH_INFO: `/v3/projects/f801adde056c45a0b05b46ed8ca76d47/users/637a011a165c4759a5bcd9df74880e17/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:15.226499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f801adde056c45a0b05b46ed8ca76d47, user_id=637a011a165c4759a5bcd9df74880e17, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:15.229231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:15.240132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1294/2571] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:15 2022] PUT /identity/v3/projects/f801adde056c45a0b05b46ed8ca76d47/users/637a011a165c4759a5bcd9df74880e17/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:15.250736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b041e3-6895-438d-ba82-a32f058dd876 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:15.251027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b041e3-6895-438d-ba82-a32f058dd876 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:15.251308 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b041e3-6895-438d-ba82-a32f058dd876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:15.279180 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-49b041e3-6895-438d-ba82-a32f058dd876 None None] MFA Rules not processed for user `637a011a165c4759a5bcd9df74880e17`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:15.280781 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-49b041e3-6895-438d-ba82-a32f058dd876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:15.356104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1278/2572] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:15.367217 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:15.368979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:15.428936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b420e68-5e36-429f-abb0-6bc64d77ef1d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:15.432625 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:15.433300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:15.433582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:15.433868 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:15.443488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:15.447754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:15.458405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1295/2573] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:15.534940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b041e3-6895-438d-ba82-a32f058dd876 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:15.547295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b041e3-6895-438d-ba82-a32f058dd876 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:15.551989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97ba7e-ad8e-43d1-b401-9c2499faecf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:15.552644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97ba7e-ad8e-43d1-b401-9c2499faecf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:15.552917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97ba7e-ad8e-43d1-b401-9c2499faecf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:15.553216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97ba7e-ad8e-43d1-b401-9c2499faecf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:15.626941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:15.633953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97ba7e-ad8e-43d1-b401-9c2499faecf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:15.636721 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97ba7e-ad8e-43d1-b401-9c2499faecf6 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:15.651447 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1279/2574] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:18.425977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:18.478506 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:18.494901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e972-34ad-447f-bbb9-fcecc13c8b9c service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:18.499813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:18.500700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:18.500978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:18.501278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] PATH_INFO: `/v3/users/95fdb9c3caea4faab18d209dbb077cdc` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:18.502510 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95fdb9c3caea4faab18d209dbb077cdc)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:18.504371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:18.550599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95fdb9c3caea4faab18d209dbb077cdc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:18.551347 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95fdb9c3caea4faab18d209dbb077cdc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:18.551942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95fdb9c3caea4faab18d209dbb077cdc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:18.565384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95fdb9c3caea4faab18d209dbb077cdc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:18.572404 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95fdb9c3caea4faab18d209dbb077cdc'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:18.587638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1296/2575] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:18 2022] DELETE /identity/v3/users/95fdb9c3caea4faab18d209dbb077cdc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:18.592763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:18.593292 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:18.593406 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:18.611192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:18.612390 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:18.677408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1280/2576] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:18.687687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:18.689228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:18.748066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3db33-6c86-4ed4-b20b-b70abd99f293 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:18.751999 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:18.752732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:18.752951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:18.753247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:18.762664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:18.764106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:18.774297 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1297/2577] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:19.139266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:19.143326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:19.156253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb3f0f-aa20-48e1-9197-a99cbfcf75c0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:19.161619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:19.162571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:19.162861 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:19.163176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] PATH_INFO: `/v3/projects/454ce4fb8c9145e983b8fa96b0b43584` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:19.164321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] RBAC: Authorizing `identity:delete_project(project_id=454ce4fb8c9145e983b8fa96b0b43584)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:19.166280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:19.216248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454ce4fb8c9145e983b8fa96b0b43584', 'request_id': 'req-e64ca514-0058-4365-9e0a-ccfc200a314d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.227561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454ce4fb8c9145e983b8fa96b0b43584', 'request_id': 'req-e64ca514-0058-4365-9e0a-ccfc200a314d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.234862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454ce4fb8c9145e983b8fa96b0b43584', 'request_id': 'req-e64ca514-0058-4365-9e0a-ccfc200a314d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.246520 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] The token cache is being invalidate because project 454ce4fb8c9145e983b8fa96b0b43584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:19.246807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.247560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.249302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1281/2578] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:19 2022] DELETE /identity/v3/projects/454ce4fb8c9145e983b8fa96b0b43584 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:19.254039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:19.266933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf008b4b-43b7-4056-9f85-0c4c2c50dccf service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:19.336079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:19.336961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:19.337269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:19.337546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] PATH_INFO: `/v3/users/637a011a165c4759a5bcd9df74880e17` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:19.338735 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=637a011a165c4759a5bcd9df74880e17)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:19.340794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:19.350645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:19.387267 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '637a011a165c4759a5bcd9df74880e17'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.388172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '637a011a165c4759a5bcd9df74880e17'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.388772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '637a011a165c4759a5bcd9df74880e17'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.402193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '637a011a165c4759a5bcd9df74880e17'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.409861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-748a4eb2-e262-4a12-aee4-6ff03caeb1a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '637a011a165c4759a5bcd9df74880e17'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.424834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1298/2579] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:19 2022] DELETE /identity/v3/users/637a011a165c4759a5bcd9df74880e17 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:19.708211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:19.709678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:19.724064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca514-0058-4365-9e0a-ccfc200a314d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:19.785237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:19.786109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:19.786392 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:19.786663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] PATH_INFO: `/v3/projects/f801adde056c45a0b05b46ed8ca76d47` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:19.787857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f801adde056c45a0b05b46ed8ca76d47)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:19.789800 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:19.820458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:19.849686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f801adde056c45a0b05b46ed8ca76d47', 'request_id': 'req-36fd18c1-8349-47de-a486-79be29bef9bd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.857837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f801adde056c45a0b05b46ed8ca76d47', 'request_id': 'req-36fd18c1-8349-47de-a486-79be29bef9bd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.864107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f801adde056c45a0b05b46ed8ca76d47', 'request_id': 'req-36fd18c1-8349-47de-a486-79be29bef9bd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.875908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] The token cache is being invalidate because project f801adde056c45a0b05b46ed8ca76d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:19.876179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.876842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-36fd18c1-8349-47de-a486-79be29bef9bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:19.878723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1282/2580] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:19 2022] DELETE /identity/v3/projects/f801adde056c45a0b05b46ed8ca76d47 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:19.883771 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc176e3-16bc-465c-916f-7a6bcfd14c45 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:19.884249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc176e3-16bc-465c-916f-7a6bcfd14c45 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:19.884367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc176e3-16bc-465c-916f-7a6bcfd14c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:19.903169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cfc176e3-16bc-465c-916f-7a6bcfd14c45 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:19.904768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cfc176e3-16bc-465c-916f-7a6bcfd14c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:19.968010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1299/2581] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:19.976096 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8d8d-e1cf-4ace-bd33-7611fa03a8a6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:19.976373 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8d8d-e1cf-4ace-bd33-7611fa03a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:19.976644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8d8d-e1cf-4ace-bd33-7611fa03a8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:19.996132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-486c8d8d-e1cf-4ace-bd33-7611fa03a8a6 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:19.997394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-486c8d8d-e1cf-4ace-bd33-7611fa03a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:20.008569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1283/2582] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:20.013477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc176e3-16bc-465c-916f-7a6bcfd14c45 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.024090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc176e3-16bc-465c-916f-7a6bcfd14c45 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.030318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.031019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.031299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.031571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.032300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:20.032605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.034265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.046247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1300/2583] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:20:20.050899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8d8d-e1cf-4ace-bd33-7611fa03a8a6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.063720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8d8d-e1cf-4ace-bd33-7611fa03a8a6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.069804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.070455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.070792 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.071000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.071683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.073600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.103326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1284/2584] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:20 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:20.108123 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.119573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e053-17ce-4372-8a2d-907888d8d1f4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.178753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.179436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.179713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.179984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.180668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.182609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.204173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1301/2585] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:20 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:20.208199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.219335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be1c39-8bb4-4034-8ac1-5df6f4aaa78e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.226187 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.226921 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.227263 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.227561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.228337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.230053 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.241947 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1285/2586] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:20.245892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.252706 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:20.256869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607c27-e4e6-480e-afc1-8afdb1e2ee64 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.262986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010c3b3-7641-4850-80a1-f504b9b9ae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.263672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010c3b3-7641-4850-80a1-f504b9b9ae9a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.264113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010c3b3-7641-4850-80a1-f504b9b9ae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.264555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010c3b3-7641-4850-80a1-f504b9b9ae9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.265354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010c3b3-7641-4850-80a1-f504b9b9ae9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.267480 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010c3b3-7641-4850-80a1-f504b9b9ae9a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.287994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1302/2587] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:20.289142 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.299755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43b365-6176-41f0-b194-de23990f46a6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.307449 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.308468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.308607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.309326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] PATH_INFO: `/v3/projects/ca04cc1fca91465397c548ebbd984337/users/2b74b62e5a404513be97728531fa8770/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.342199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca04cc1fca91465397c548ebbd984337, user_id=2b74b62e5a404513be97728531fa8770, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.344838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.362078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1286/2588] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:20 2022] PUT /identity/v3/projects/ca04cc1fca91465397c548ebbd984337/users/2b74b62e5a404513be97728531fa8770/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:20.376288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34efad6-6504-411e-8437-3bbd06dbad6c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.376577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34efad6-6504-411e-8437-3bbd06dbad6c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.376884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34efad6-6504-411e-8437-3bbd06dbad6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.403076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-c34efad6-6504-411e-8437-3bbd06dbad6c None None] MFA Rules not processed for user `2b74b62e5a404513be97728531fa8770`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:20.404630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c34efad6-6504-411e-8437-3bbd06dbad6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:20.470040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1303/2589] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:20.474600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.539639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e77f1e-e023-41a7-8c77-becc41b92f25 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.544616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.545317 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.545596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.545868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.546570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.548501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.575245 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1287/2590] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:20 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:20.578938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34efad6-6504-411e-8437-3bbd06dbad6c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.589587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34efad6-6504-411e-8437-3bbd06dbad6c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.645700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.646418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.646831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.647068 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.647850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.649814 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.681348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1304/2591] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:20 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:20.685391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.696796 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b778b1-b5a0-4499-a689-2fa23a33b345 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.702709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.703443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.703810 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.704201 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.704950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.709247 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.719664 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1288/2592] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:20.724063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.736250 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9b057-4a82-4920-b2be-9dab6bef0b8a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.740784 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.741720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.741998 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.742271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] PATH_INFO: `/v3/projects/7b1b9dcec5e34d4499488a33f36cbc8a/users/bf743ed878fd417682d2dab34aa089c4/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.760465 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1b9dcec5e34d4499488a33f36cbc8a, user_id=bf743ed878fd417682d2dab34aa089c4, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.763841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.780169 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1305/2593] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:20 2022] PUT /identity/v3/projects/7b1b9dcec5e34d4499488a33f36cbc8a/users/bf743ed878fd417682d2dab34aa089c4/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:20.784322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.794544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee7f5f-792e-4640-bcaa-e25a6c3aa9ed admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.806977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:20.858621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa930841-71ab-41cd-86fb-f1bac4c730d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.859217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa930841-71ab-41cd-86fb-f1bac4c730d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.859498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa930841-71ab-41cd-86fb-f1bac4c730d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.859753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa930841-71ab-41cd-86fb-f1bac4c730d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.860474 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa930841-71ab-41cd-86fb-f1bac4c730d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.862105 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa930841-71ab-41cd-86fb-f1bac4c730d9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.872533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1289/2594] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:20.876977 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:20.887880 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb96e9b-6de4-4d63-8fa4-b81bdba8ae57 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:20.893681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:20.894575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.894855 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.895127 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] PATH_INFO: `/v3/projects/7b1b9dcec5e34d4499488a33f36cbc8a/users/bf743ed878fd417682d2dab34aa089c4/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.897492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1b9dcec5e34d4499488a33f36cbc8a, user_id=bf743ed878fd417682d2dab34aa089c4, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:20.900140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:20.911435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1306/2595] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:20 2022] PUT /identity/v3/projects/7b1b9dcec5e34d4499488a33f36cbc8a/users/bf743ed878fd417682d2dab34aa089c4/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:20.921603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:20.921874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:20.922150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:20.948327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] MFA Rules not processed for user `bf743ed878fd417682d2dab34aa089c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:20.949603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:21.028953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1290/2596] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:21.041032 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:21.043031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:21.089755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:21.112246 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3e7ca-0b6a-4410-9090-7f937fa601d3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:21.116406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:21.117284 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:21.117668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:21.118029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:21.180342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:21.181859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:21.191074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1307/2597] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:23.558840 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:23.560662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:23.620787 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1dd9a-a3a1-472f-a7d4-b38477698994 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:23.624799 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:23.625478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:23.625760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:23.626037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:23.627893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:23.687130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:23.692614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:23.695641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:23.707758 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1291/2598] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:25.439831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:25.441468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:25.453930 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1718ab4-f501-4e44-b587-a90aca28c40f service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:25.457758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6852b-cb7d-478f-aaef-408462e10204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:25.458449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6852b-cb7d-478f-aaef-408462e10204 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:25.458736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6852b-cb7d-478f-aaef-408462e10204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:25.459001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6852b-cb7d-478f-aaef-408462e10204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:25.469352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6852b-cb7d-478f-aaef-408462e10204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:25.470801 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6852b-cb7d-478f-aaef-408462e10204 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:25.481379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1308/2599] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:27.229996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:27.231666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:27.301878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0de02-adc7-4c12-8777-b110e5e0835f service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:27.307174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:27.308109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:27.308381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:27.308687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] PATH_INFO: `/v3/users/366e0d94a93c43958349e333a1edd060` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:27.309962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] RBAC: Authorizing `identity:delete_user(user_id=366e0d94a93c43958349e333a1edd060)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:27.312588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:27.373869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366e0d94a93c43958349e333a1edd060'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:27.374505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366e0d94a93c43958349e333a1edd060'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:27.386757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366e0d94a93c43958349e333a1edd060'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:27.387438 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366e0d94a93c43958349e333a1edd060'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:27.394499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366e0d94a93c43958349e333a1edd060'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:27.408522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1292/2600] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:27 2022] DELETE /identity/v3/users/366e0d94a93c43958349e333a1edd060 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:27.413500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:27.414015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:27.414547 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:27.433665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:27.435161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:27.507747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1309/2601] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:27.520414 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:27.523329 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:27.589591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01126e66-3a38-4405-a162-2c2645dc496f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:27.593857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756257d6-b2f6-4b40-a95d-e43b21050d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:27.594581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756257d6-b2f6-4b40-a95d-e43b21050d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:27.594859 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756257d6-b2f6-4b40-a95d-e43b21050d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:27.595169 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756257d6-b2f6-4b40-a95d-e43b21050d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:27.605027 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756257d6-b2f6-4b40-a95d-e43b21050d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:27.606563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756257d6-b2f6-4b40-a95d-e43b21050d3e service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:27.616192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1293/2602] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:27.906720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:27.908302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:27.924191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c4d8d-c011-44a5-ad86-781074923f6f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:27.930045 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:27.931657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:27.932190 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:27.932520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] PATH_INFO: `/v3/projects/00662d58033946ffb970e3892aae9e3f` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:27.934128 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00662d58033946ffb970e3892aae9e3f)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:27.937010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:27.990715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00662d58033946ffb970e3892aae9e3f', 'request_id': 'req-9da98a20-b040-4893-96d4-6adf688acf95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:28.010019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00662d58033946ffb970e3892aae9e3f', 'request_id': 'req-9da98a20-b040-4893-96d4-6adf688acf95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:28.018676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00662d58033946ffb970e3892aae9e3f', 'request_id': 'req-9da98a20-b040-4893-96d4-6adf688acf95'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:28.032236 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:28.042377 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] The token cache is being invalidate because project 00662d58033946ffb970e3892aae9e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:28.042773 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:28.044162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-9da98a20-b040-4893-96d4-6adf688acf95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:28.047224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1310/2603] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:27 2022] DELETE /identity/v3/projects/00662d58033946ffb970e3892aae9e3f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:28.052154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382f4f-c870-45ce-83fb-514237c6b3e4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.052585 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382f4f-c870-45ce-83fb-514237c6b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.052978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382f4f-c870-45ce-83fb-514237c6b3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.078226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-52382f4f-c870-45ce-83fb-514237c6b3e4 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:28.079576 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-52382f4f-c870-45ce-83fb-514237c6b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:28.097026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:28.197885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1294/2604] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:28.206230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3965ba-5850-4d6f-85c1-a3f847b6c017 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.206744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3965ba-5850-4d6f-85c1-a3f847b6c017 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.207104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3965ba-5850-4d6f-85c1-a3f847b6c017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.235748 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9e3965ba-5850-4d6f-85c1-a3f847b6c017 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:28.237019 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9e3965ba-5850-4d6f-85c1-a3f847b6c017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:28.249166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1311/2605] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:28.251600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52382f4f-c870-45ce-83fb-514237c6b3e4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:28.262865 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52382f4f-c870-45ce-83fb-514237c6b3e4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:28.272455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:28.274224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.274514 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.274823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.275569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:28.275910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:28.277606 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:28.291814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1295/2606] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:28.296679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3965ba-5850-4d6f-85c1-a3f847b6c017 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:28.317483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3965ba-5850-4d6f-85c1-a3f847b6c017 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:28.325902 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:28.326722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.327074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.327412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.328254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:28.330429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:28.365113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1312/2607] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:28 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:28.366376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:28.378037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c020-330d-4f6d-9ebc-76e38bb32393 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:28.447945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:28.448815 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.449186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.449567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.450371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:28.454980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:28.480358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1296/2608] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:28 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:28.484826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:28.515939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d53c1-2068-43ab-b8cb-46b6133ca6f8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:28.531938 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:28.532683 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.533261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.533762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.534589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:28.536464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:28.591841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1313/2609] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:28.596222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:28.609568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ba92b-7c2e-489a-90a4-44f9c10d87a8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:28.619619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07b208-9dc0-479a-807a-d1610f5b0831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:28.621451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b208-9dc0-479a-807a-d1610f5b0831 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.622111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b208-9dc0-479a-807a-d1610f5b0831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.623117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b208-9dc0-479a-807a-d1610f5b0831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.624678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07b208-9dc0-479a-807a-d1610f5b0831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:28.627667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07b208-9dc0-479a-807a-d1610f5b0831 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:28.652422 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1297/2610] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:28.654690 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:28.668519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8254be-61d8-4ebd-bfb7-03b5a3b937b5 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:28.675836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:28.676770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.677074 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.677378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] PATH_INFO: `/v3/projects/74a1a48260954e2598bfe1b43874d491/users/647e3bf3d2c849348c8071f1224c8d34/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.699103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a1a48260954e2598bfe1b43874d491, user_id=647e3bf3d2c849348c8071f1224c8d34, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:28.703046 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:28.727157 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1314/2611] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:28 2022] PUT /identity/v3/projects/74a1a48260954e2598bfe1b43874d491/users/647e3bf3d2c849348c8071f1224c8d34/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:28.740092 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.740772 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.741802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.779340 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] MFA Rules not processed for user `647e3bf3d2c849348c8071f1224c8d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:28.780728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:28.858672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1298/2612] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:28.875104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:28.877856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:28.947911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba55a5-9a72-4795-9ca4-692a683ef617 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:28.952864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995f424-50c0-4f2f-9cdc-7476d207cea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:28.953713 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995f424-50c0-4f2f-9cdc-7476d207cea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:28.954086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995f424-50c0-4f2f-9cdc-7476d207cea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:28.954471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995f424-50c0-4f2f-9cdc-7476d207cea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:28.964238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995f424-50c0-4f2f-9cdc-7476d207cea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:28.965833 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995f424-50c0-4f2f-9cdc-7476d207cea3 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:28.978992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1315/2613] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:30.590298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:30.591942 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:30.607094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:30.671150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb4b0a-7ce1-4a85-8e27-c4ab43b4cfe5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:30.676588 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:30.677679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:30.678049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:30.678327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] PATH_INFO: `/v3/users/2b74b62e5a404513be97728531fa8770` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:30.679860 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b74b62e5a404513be97728531fa8770)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:30.682350 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:30.741524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b74b62e5a404513be97728531fa8770'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:30.742233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b74b62e5a404513be97728531fa8770'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:30.755567 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b74b62e5a404513be97728531fa8770'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:30.756233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b74b62e5a404513be97728531fa8770'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:30.763889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b74b62e5a404513be97728531fa8770'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:30.784306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1299/2614] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:30 2022] DELETE /identity/v3/users/2b74b62e5a404513be97728531fa8770 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:30.789012 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e046031-7d04-49b4-8324-80856a3f30bc None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:30.789520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e046031-7d04-49b4-8324-80856a3f30bc None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:30.789636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e046031-7d04-49b4-8324-80856a3f30bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:30.807224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0e046031-7d04-49b4-8324-80856a3f30bc None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:30.808399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0e046031-7d04-49b4-8324-80856a3f30bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:30.881509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1316/2615] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:30.894087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:30.898271 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:30.987202 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125651a-28e1-4264-9389-3a75e4674346 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:30.991681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:30.992347 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:30.992631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:30.992918 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:31.008163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:31.009666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:31.024460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1300/2616] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:31.245155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e046031-7d04-49b4-8324-80856a3f30bc None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:31.264041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e046031-7d04-49b4-8324-80856a3f30bc None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:31.267829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:31.268487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:31.268951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:31.269270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:31.271733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:31.287911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:31.289580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:31.302151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1317/2617] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:31.462081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:31.463578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:31.478483 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278f6ce-2257-46cb-bdea-b3eaf6c5ac82 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:31.483551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:31.484524 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:31.484877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:31.485262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] PATH_INFO: `/v3/projects/ca04cc1fca91465397c548ebbd984337` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:31.486538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca04cc1fca91465397c548ebbd984337)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:31.488591 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:31.526824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:31.546236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca04cc1fca91465397c548ebbd984337', 'request_id': 'req-8b41f6af-ad08-4060-8d48-ee0cfc56e267'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.554533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca04cc1fca91465397c548ebbd984337', 'request_id': 'req-8b41f6af-ad08-4060-8d48-ee0cfc56e267'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.561663 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca04cc1fca91465397c548ebbd984337', 'request_id': 'req-8b41f6af-ad08-4060-8d48-ee0cfc56e267'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.572600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] The token cache is being invalidate because project ca04cc1fca91465397c548ebbd984337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:31.572868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.573685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.576070 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1301/2618] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:31 2022] DELETE /identity/v3/projects/ca04cc1fca91465397c548ebbd984337 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:31.579787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:31.590553 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995dd17-9178-4501-a093-cde102c24f54 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:31.615911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:31.651732 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:31.652781 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:31.653078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:31.653365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] PATH_INFO: `/v3/users/bf743ed878fd417682d2dab34aa089c4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:31.655010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf743ed878fd417682d2dab34aa089c4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:31.656924 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:31.708038 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf743ed878fd417682d2dab34aa089c4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.708685 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf743ed878fd417682d2dab34aa089c4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.709303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf743ed878fd417682d2dab34aa089c4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.726700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf743ed878fd417682d2dab34aa089c4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.734148 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7fc6c1e0-8075-4605-aec5-fba6b4feeeee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf743ed878fd417682d2dab34aa089c4'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:31.749208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1318/2619] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:31 2022] DELETE /identity/v3/users/bf743ed878fd417682d2dab34aa089c4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:32.053380 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:32.054982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:32.082262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41f6af-ad08-4060-8d48-ee0cfc56e267 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:32.149662 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:32.150695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.150980 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.151253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] PATH_INFO: `/v3/projects/7b1b9dcec5e34d4499488a33f36cbc8a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.152432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1b9dcec5e34d4499488a33f36cbc8a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:32.154854 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:32.213008 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1b9dcec5e34d4499488a33f36cbc8a', 'request_id': 'req-63288763-0689-4d9c-b81d-60510da499bf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:32.226820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1b9dcec5e34d4499488a33f36cbc8a', 'request_id': 'req-63288763-0689-4d9c-b81d-60510da499bf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:32.235377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1b9dcec5e34d4499488a33f36cbc8a', 'request_id': 'req-63288763-0689-4d9c-b81d-60510da499bf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:32.248145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] The token cache is being invalidate because project 7b1b9dcec5e34d4499488a33f36cbc8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:32.248549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:32.249342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-63288763-0689-4d9c-b81d-60510da499bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:32.251167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1302/2620] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:32 2022] DELETE /identity/v3/projects/7b1b9dcec5e34d4499488a33f36cbc8a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:32.257590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413341aa-0dc2-4332-9705-19370f9eabae None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.258119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413341aa-0dc2-4332-9705-19370f9eabae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.258332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413341aa-0dc2-4332-9705-19370f9eabae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.279374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-413341aa-0dc2-4332-9705-19370f9eabae None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:32.280692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-413341aa-0dc2-4332-9705-19370f9eabae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:32.366048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1319/2621] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:32.374640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee02c-b13f-4b55-8f80-9b1e34adb02e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.374895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee02c-b13f-4b55-8f80-9b1e34adb02e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.375172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee02c-b13f-4b55-8f80-9b1e34adb02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.403176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d37ee02c-b13f-4b55-8f80-9b1e34adb02e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:32.404786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d37ee02c-b13f-4b55-8f80-9b1e34adb02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:32.418374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413341aa-0dc2-4332-9705-19370f9eabae None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:32.423394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1303/2622] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:32.429629 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413341aa-0dc2-4332-9705-19370f9eabae None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:32.444959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:32.446009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.446380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.446746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.447603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:32.448001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:32.449769 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:32.466582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1320/2623] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:20:32.474755 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ee02c-b13f-4b55-8f80-9b1e34adb02e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:32.498841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ee02c-b13f-4b55-8f80-9b1e34adb02e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:32.506915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:32.507596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.507874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.508141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.508839 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:32.510908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:32.554399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1304/2624] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:32 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:32.558669 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:32.569452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f28ea-c5d7-47e0-9dca-5469a7eaed6e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:32.604936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:32.662119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:32.663153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.663436 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.663826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.664596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:32.666765 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:32.701374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1321/2625] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:32 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:32.705407 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:32.717717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbdab6-4fd5-4510-a6a9-e01c95196de2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:32.722538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:32.723484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.723687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.725094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.725094 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:32.726730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:32.738030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1305/2626] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:32.742381 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:32.753692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cf33a-73d8-473b-9096-4a3c48ae2b12 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:32.758252 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078666d-9067-4d4f-9747-a3d28a783edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:32.758919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078666d-9067-4d4f-9747-a3d28a783edf admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.759202 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078666d-9067-4d4f-9747-a3d28a783edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.759477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078666d-9067-4d4f-9747-a3d28a783edf admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.760208 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078666d-9067-4d4f-9747-a3d28a783edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:32.761856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078666d-9067-4d4f-9747-a3d28a783edf admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:32.775593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1322/2627] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:32.776927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:32.788928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610ad2-82e1-4a29-8d58-7f58552bfbd9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:32.793909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:32.794971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.795242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.795510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] PATH_INFO: `/v3/projects/aed7fa3810f3443a9f4972b9b6442ca2/users/b1ead1e7ccfc4ccdb60d8b4877ed7f64/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.798049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:32.813910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed7fa3810f3443a9f4972b9b6442ca2, user_id=b1ead1e7ccfc4ccdb60d8b4877ed7f64, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:32.819174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:32.834714 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1306/2628] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:32 2022] PUT /identity/v3/projects/aed7fa3810f3443a9f4972b9b6442ca2/users/b1ead1e7ccfc4ccdb60d8b4877ed7f64/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:32.847322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf271d1-971f-4617-ae46-c94efb41ac8f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:32.847905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf271d1-971f-4617-ae46-c94efb41ac8f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:32.848031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf271d1-971f-4617-ae46-c94efb41ac8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:32.877090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5bf271d1-971f-4617-ae46-c94efb41ac8f None None] MFA Rules not processed for user `b1ead1e7ccfc4ccdb60d8b4877ed7f64`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:32.878671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5bf271d1-971f-4617-ae46-c94efb41ac8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:32.962420 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1323/2629] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:32.967620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.058678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ee068-0ed0-434b-a8ba-adb40c106627 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.064311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.065005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.065321 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.065605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.066516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.068404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.100819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1307/2630] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:32 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:33.104918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf271d1-971f-4617-ae46-c94efb41ac8f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.116561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf271d1-971f-4617-ae46-c94efb41ac8f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.185505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.186400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.186725 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.186951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.189115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.191408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.215285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1324/2631] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:33 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:20:33.219853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.231746 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834ddb2-d4ea-4823-ab4b-de94693e4591 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.236851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.237771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.238118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.238472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.239284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.240967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.252337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1308/2632] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:33.256712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.267291 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0fab4-15d6-46b5-9ed6-bfd496e183d2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.272124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ad33f-a688-4342-88b5-139ae2166bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.272822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ad33f-a688-4342-88b5-139ae2166bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.273140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ad33f-a688-4342-88b5-139ae2166bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.273543 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ad33f-a688-4342-88b5-139ae2166bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.274992 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ad33f-a688-4342-88b5-139ae2166bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.276663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ad33f-a688-4342-88b5-139ae2166bd5 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.287861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1325/2633] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:33.292102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.302409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549ede9-3d54-4c4c-b04e-d0a2aad8301e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.307454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.308538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.308816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.309101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] PATH_INFO: `/v3/projects/2e69e489c0874765aab8316f992688e2/users/be18d5feb8a24ea388d1183edd7587fb/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.328231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e69e489c0874765aab8316f992688e2, user_id=be18d5feb8a24ea388d1183edd7587fb, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.330948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.345593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1309/2634] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:33 2022] PUT /identity/v3/projects/2e69e489c0874765aab8316f992688e2/users/be18d5feb8a24ea388d1183edd7587fb/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:33.358035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2239905-4ac3-471f-8fc9-ca26593e7940 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.358445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2239905-4ac3-471f-8fc9-ca26593e7940 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.358724 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2239905-4ac3-471f-8fc9-ca26593e7940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.386183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e2239905-4ac3-471f-8fc9-ca26593e7940 None None] MFA Rules not processed for user `be18d5feb8a24ea388d1183edd7587fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:33.387489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e2239905-4ac3-471f-8fc9-ca26593e7940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:33.396283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:33.476655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1326/2635] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:33.481226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.564468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bd8d3-5a52-4610-b298-94eaf6931f87 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.569386 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.569958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.570236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.570507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.571548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.573114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.612144 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1310/2636] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:33 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:20:33.617330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2239905-4ac3-471f-8fc9-ca26593e7940 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.630866 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2239905-4ac3-471f-8fc9-ca26593e7940 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.739232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.740374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.742009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.742417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.744457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.750572 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.790679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1327/2637] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:33 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:33.792526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.808773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2acec-e5d5-4ceb-939b-708f3595a544 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.828593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.828593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.828593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.828593 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.829006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.829006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.852723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.853442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1311/2638] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:33.869525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a115-0c4e-4a23-a617-5464b4b33e35 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.879674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:33.880751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:33.881071 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:33.881360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] PATH_INFO: `/v3/projects/bc1d1d7f144849e2b253f23948d52933/users/b989c61d94c34fa9acc2d5373b603e9e/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:33.900125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1d1d7f144849e2b253f23948d52933, user_id=b989c61d94c34fa9acc2d5373b603e9e, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:33.908883 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:33.929228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1328/2639] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:33 2022] PUT /identity/v3/projects/bc1d1d7f144849e2b253f23948d52933/users/b989c61d94c34fa9acc2d5373b603e9e/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:33.933487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:33.947702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b05e4-91d1-42a5-9cbb-46aa74852bd6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:33.952404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:34.078506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d532cf1-19d5-4a98-9915-e77e5660c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:34.078506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d532cf1-19d5-4a98-9915-e77e5660c17c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:34.079338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d532cf1-19d5-4a98-9915-e77e5660c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:34.079338 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d532cf1-19d5-4a98-9915-e77e5660c17c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:34.079753 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d532cf1-19d5-4a98-9915-e77e5660c17c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:34.088554 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d532cf1-19d5-4a98-9915-e77e5660c17c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:34.102083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1312/2640] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:34.106358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:34.121342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22984822-29b4-4a8b-897e-20355c49fe60 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:34.127434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:34.131712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:34.132455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:34.133177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] PATH_INFO: `/v3/projects/bc1d1d7f144849e2b253f23948d52933/users/b989c61d94c34fa9acc2d5373b603e9e/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:34.135985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1d1d7f144849e2b253f23948d52933, user_id=b989c61d94c34fa9acc2d5373b603e9e, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:34.139590 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:34.152345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1329/2641] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:34 2022] PUT /identity/v3/projects/bc1d1d7f144849e2b253f23948d52933/users/b989c61d94c34fa9acc2d5373b603e9e/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:34.165302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:34.165885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:34.166339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:34.197563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] MFA Rules not processed for user `b989c61d94c34fa9acc2d5373b603e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:34.198794 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:34.299306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1313/2642] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:34.311070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:34.313035 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:34.380173 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597dfa3-3cb9-4738-b406-b1ab56c16940 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:34.387610 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:34.388418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:34.388790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:34.389441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:34.406810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:34.456832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:34.458519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:34.470242 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1330/2643] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:35.085631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:35.087249 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:35.176314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0624b37-4e8c-4873-b0a1-61013782f13c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:35.190889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:35.191806 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:35.192480 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:35.192910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:35.195719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:35.274676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:35.319439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:35.322134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:35.338256 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1314/2644] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:36.153240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:36.170494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6129c6-8851-4551-87d4-e498adb4070e service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:36.174966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:36.175626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:36.175907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:36.176180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:36.253692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:36.255571 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:36.267866 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1331/2645] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:36.937884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:36.948497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585dd1d-4943-4dcd-a4c3-f42c807400c5 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:36.952503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba4f20-efe5-4f8a-ad16-b909065ad484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:36.953268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba4f20-efe5-4f8a-ad16-b909065ad484 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:36.953641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba4f20-efe5-4f8a-ad16-b909065ad484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:36.953985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba4f20-efe5-4f8a-ad16-b909065ad484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:36.964454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba4f20-efe5-4f8a-ad16-b909065ad484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:36.965958 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba4f20-efe5-4f8a-ad16-b909065ad484 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:36.976353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1315/2646] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:40.155689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:40.157262 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:40.172213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782a3eb-d139-47cf-963b-97bab8bcf75d service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:40.177502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:40.178503 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:40.178953 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:40.179232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] PATH_INFO: `/v3/users/647e3bf3d2c849348c8071f1224c8d34` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:40.180455 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=647e3bf3d2c849348c8071f1224c8d34)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:40.182452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:40.300041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '647e3bf3d2c849348c8071f1224c8d34'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:40.300768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '647e3bf3d2c849348c8071f1224c8d34'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:40.301392 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '647e3bf3d2c849348c8071f1224c8d34'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:40.336064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '647e3bf3d2c849348c8071f1224c8d34'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:40.357700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '647e3bf3d2c849348c8071f1224c8d34'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:40.386558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1332/2647] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:40 2022] DELETE /identity/v3/users/647e3bf3d2c849348c8071f1224c8d34 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:40.391090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:40.391916 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:40.392358 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:40.420039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:40.423132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:40.518252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1316/2648] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:40.529276 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:40.531406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:40.606846 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:40.659585 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5f88-6779-43b1-9c93-3bd254288c8c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:40.670534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:40.672380 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:40.675182 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:40.675518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:40.701632 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:40.703958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:40.724791 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1333/2649] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:40.835838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:40.837486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:40.865749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9bcc6-45bf-47ac-b021-3001c53cad8e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:40.879306 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:40.880355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:40.880698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:40.881611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] PATH_INFO: `/v3/users/516ee78e2f9c43b195ad64b7a1d68624` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:40.886012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=516ee78e2f9c43b195ad64b7a1d68624)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:40.889651 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:40.982441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:40.984508 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.020487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526c8c1-f4e4-498e-8abc-4ad439415d8b service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.040989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.041917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.042199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.042471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] PATH_INFO: `/v3/users/b1ead1e7ccfc4ccdb60d8b4877ed7f64` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.044253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ead1e7ccfc4ccdb60d8b4877ed7f64)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:41.049957 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516ee78e2f9c43b195ad64b7a1d68624'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.051353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:41.057528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516ee78e2f9c43b195ad64b7a1d68624'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.085826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516ee78e2f9c43b195ad64b7a1d68624'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.086826 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516ee78e2f9c43b195ad64b7a1d68624'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.098342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516ee78e2f9c43b195ad64b7a1d68624'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.126831 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1317/2650] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:40 2022] DELETE /identity/v3/users/516ee78e2f9c43b195ad64b7a1d68624 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:41.131258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:41.134310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.160290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ead1e7ccfc4ccdb60d8b4877ed7f64'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.161446 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ead1e7ccfc4ccdb60d8b4877ed7f64'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.162556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ead1e7ccfc4ccdb60d8b4877ed7f64'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.184193 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ead1e7ccfc4ccdb60d8b4877ed7f64'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.196375 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ead1e7ccfc4ccdb60d8b4877ed7f64'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.198886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:41.221258 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1334/2651] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:40 2022] DELETE /identity/v3/users/b1ead1e7ccfc4ccdb60d8b4877ed7f64 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:41.222762 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:41.225425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.258093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd763f61-f7f5-44cf-9b58-73828e3a92cb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.336043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354534c-7333-41f7-8925-40c7fe67cf74 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.339582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.340518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.340913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.341215 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] PATH_INFO: `/v3/users/53278c1c534344978ca40d7e2313c02c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.343168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53278c1c534344978ca40d7e2313c02c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:41.346257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:41.383740 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fe662-b342-47b1-8143-d550d5c28270 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.386113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fe662-b342-47b1-8143-d550d5c28270 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.389095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fe662-b342-47b1-8143-d550d5c28270 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.390577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fe662-b342-47b1-8143-d550d5c28270 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.393103 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f36fe662-b342-47b1-8143-d550d5c28270 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.410445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fe662-b342-47b1-8143-d550d5c28270 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:41.413688 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fe662-b342-47b1-8143-d550d5c28270 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:41.418772 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:41.425323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53278c1c534344978ca40d7e2313c02c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.426529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53278c1c534344978ca40d7e2313c02c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.428279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53278c1c534344978ca40d7e2313c02c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.431383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1318/2652] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:41.433395 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.433857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.434283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.446745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53278c1c534344978ca40d7e2313c02c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.461989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:41.464272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53278c1c534344978ca40d7e2313c02c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.464552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.479012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1319/2653] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:41.480078 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:41.483001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.494322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1335/2654] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:41 2022] DELETE /identity/v3/users/53278c1c534344978ca40d7e2313c02c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:41.494322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:41.498029 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.500153 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50b1bd-e903-40f8-b5d5-a74d555e1e66 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.584716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c435f9a-d9c6-4cdc-950f-27e1c6ef9740 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.587157 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.588638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.588929 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.589281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] PATH_INFO: `/v3/projects/74a1a48260954e2598bfe1b43874d491` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.590502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a1a48260954e2598bfe1b43874d491)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:41.591696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.592694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:41.593201 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.593688 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.594179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] PATH_INFO: `/v3/projects/04a2e6cc3cbd4691a33722b5d058e82b` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.595814 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a2e6cc3cbd4691a33722b5d058e82b)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:41.598011 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:41.651798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a1a48260954e2598bfe1b43874d491', 'request_id': 'req-15cd475f-395d-461d-b2b9-927d15f7a222'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.657612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a2e6cc3cbd4691a33722b5d058e82b', 'request_id': 'req-e7d2b0b2-6da9-40c4-98ac-5286351d189e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.663143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a1a48260954e2598bfe1b43874d491', 'request_id': 'req-15cd475f-395d-461d-b2b9-927d15f7a222'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.664870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a2e6cc3cbd4691a33722b5d058e82b', 'request_id': 'req-e7d2b0b2-6da9-40c4-98ac-5286351d189e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.673210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a2e6cc3cbd4691a33722b5d058e82b', 'request_id': 'req-e7d2b0b2-6da9-40c4-98ac-5286351d189e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.673760 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a1a48260954e2598bfe1b43874d491', 'request_id': 'req-15cd475f-395d-461d-b2b9-927d15f7a222'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.679388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:41.688620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] The token cache is being invalidate because project 74a1a48260954e2598bfe1b43874d491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:41.688917 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.689969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.691269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] The token cache is being invalidate because project 04a2e6cc3cbd4691a33722b5d058e82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:41.691586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.692079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1320/2655] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:41 2022] DELETE /identity/v3/projects/74a1a48260954e2598bfe1b43874d491 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:41.692891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:41.695138 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.696254 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.700716 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1336/2656] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:41 2022] DELETE /identity/v3/projects/04a2e6cc3cbd4691a33722b5d058e82b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:41.701313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:41.702714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.770283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd475f-395d-461d-b2b9-927d15f7a222 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.786461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:41.804398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2b0b2-6da9-40c4-98ac-5286351d189e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.811684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.812911 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.813282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.813665 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] PATH_INFO: `/v3/projects/2ed408b23e4c48a5861e542d02e0ee64` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.814988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed408b23e4c48a5861e542d02e0ee64)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:41.816986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:41.835686 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.836384 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.836667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.836943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.839542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.856801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:41.858962 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:41.872146 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed408b23e4c48a5861e542d02e0ee64', 'request_id': 'req-0e2362f9-c53b-451b-88ad-c97418606c38'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.872741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1321/2657] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:41.875133 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:41.878333 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.880972 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed408b23e4c48a5861e542d02e0ee64', 'request_id': 'req-0e2362f9-c53b-451b-88ad-c97418606c38'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.888118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed408b23e4c48a5861e542d02e0ee64', 'request_id': 'req-0e2362f9-c53b-451b-88ad-c97418606c38'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.904021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b131-befc-4158-b187-9a9ca1189b4f service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:41.905201 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] The token cache is being invalidate because project 2ed408b23e4c48a5861e542d02e0ee64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:41.905573 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.906270 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-0e2362f9-c53b-451b-88ad-c97418606c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:41.911350 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1337/2658] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:41 2022] DELETE /identity/v3/projects/2ed408b23e4c48a5861e542d02e0ee64 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:41.913438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e67f8-8be0-4aa1-83c2-ab0567463af1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.913795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e67f8-8be0-4aa1-83c2-ab0567463af1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.914460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e67f8-8be0-4aa1-83c2-ab0567463af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.938947 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-364e67f8-8be0-4aa1-83c2-ab0567463af1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:41.940520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-364e67f8-8be0-4aa1-83c2-ab0567463af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:41.993276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:41.994709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:41.995059 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:41.995464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] PATH_INFO: `/v3/users/3404aa16454b4735a63758f87eb920a8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:41.998156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3404aa16454b4735a63758f87eb920a8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.000695 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.021630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1338/2659] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:42.029751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246c976-5278-42cf-bb4a-ff7bec0d2afa None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.030044 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246c976-5278-42cf-bb4a-ff7bec0d2afa None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.030349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246c976-5278-42cf-bb4a-ff7bec0d2afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.052417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0246c976-5278-42cf-bb4a-ff7bec0d2afa None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:42.053792 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0246c976-5278-42cf-bb4a-ff7bec0d2afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.061666 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3404aa16454b4735a63758f87eb920a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.062673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3404aa16454b4735a63758f87eb920a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.068110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1339/2660] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:42.069238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246c976-5278-42cf-bb4a-ff7bec0d2afa None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.070757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=9,keystone:SELECT=91 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:42.081551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246c976-5278-42cf-bb4a-ff7bec0d2afa None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.088932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.089975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.091303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.091303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.091624 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3404aa16454b4735a63758f87eb920a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.092280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:42.092835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.095039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.097668 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3404aa16454b4735a63758f87eb920a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.107660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3404aa16454b4735a63758f87eb920a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.109432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1340/2661] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:42.114163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.116286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.127283 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1322/2662] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:41 2022] DELETE /identity/v3/users/3404aa16454b4735a63758f87eb920a8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:42.148776 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e29e2-6770-4ebd-a3d6-4a46ea70f763 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.243025 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.245240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.249449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.250349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.250631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.250901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.251597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.259910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.266138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:42.274555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d588-6aed-4356-9a4f-5f70da5b34c9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.279870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.280759 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.281185 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.281462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] PATH_INFO: `/v3/projects/aed7fa3810f3443a9f4972b9b6442ca2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.283393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed7fa3810f3443a9f4972b9b6442ca2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.285365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.315798 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1341/2663] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/projects => generated 371 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:42.319892 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.349659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed7fa3810f3443a9f4972b9b6442ca2', 'request_id': 'req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.367316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed7fa3810f3443a9f4972b9b6442ca2', 'request_id': 'req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.378506 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed7fa3810f3443a9f4972b9b6442ca2', 'request_id': 'req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.393636 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] The token cache is being invalidate because project aed7fa3810f3443a9f4972b9b6442ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:42.394291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.395028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.403000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1323/2664] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:42 2022] DELETE /identity/v3/projects/aed7fa3810f3443a9f4972b9b6442ca2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:42.404556 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a84bb-c41c-42b2-acc0-90cc3d960e61 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.405736 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.408131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.468039 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.469064 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.469867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.469867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.470318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.472268 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.485104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca53a5c-ca6c-47b3-9db1-9821c0bb8700 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.491696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.492571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.492850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.493152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] PATH_INFO: `/v3/projects/dbaa095c7963427f9a8cf59b1dd023e2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.494344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbaa095c7963427f9a8cf59b1dd023e2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.496250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.496990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1342/2665] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:42.498197 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.499885 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.511811 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027f0a8-08c6-4de9-a3f6-6fc4ee220050 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.519248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.520277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.520600 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.521440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] PATH_INFO: `/v3/users/be18d5feb8a24ea388d1183edd7587fb` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.522975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be18d5feb8a24ea388d1183edd7587fb)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.524990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.556561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbaa095c7963427f9a8cf59b1dd023e2', 'request_id': 'req-d51640d4-0a90-4619-83c7-a063a61ed34f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.565750 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbaa095c7963427f9a8cf59b1dd023e2', 'request_id': 'req-d51640d4-0a90-4619-83c7-a063a61ed34f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.567344 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:42.575919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbaa095c7963427f9a8cf59b1dd023e2', 'request_id': 'req-d51640d4-0a90-4619-83c7-a063a61ed34f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.584778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be18d5feb8a24ea388d1183edd7587fb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.585407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be18d5feb8a24ea388d1183edd7587fb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.586116 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be18d5feb8a24ea388d1183edd7587fb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.590563 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] The token cache is being invalidate because project dbaa095c7963427f9a8cf59b1dd023e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:42.591062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.592692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.594354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1324/2666] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:42 2022] DELETE /identity/v3/projects/dbaa095c7963427f9a8cf59b1dd023e2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:42.598158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.599493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be18d5feb8a24ea388d1183edd7587fb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.599978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.614322 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4d81f4ef-feb2-4245-a346-8a71fb00ffd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be18d5feb8a24ea388d1183edd7587fb'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:42.618926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:42.636664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1343/2667] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:42 2022] DELETE /identity/v3/users/be18d5feb8a24ea388d1183edd7587fb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:42.638212 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cf346-2b10-426c-8478-536b2070cf67 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.638652 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cf346-2b10-426c-8478-536b2070cf67 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.639081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cf346-2b10-426c-8478-536b2070cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.659799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-894cf346-2b10-426c-8478-536b2070cf67 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:42.661328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-894cf346-2b10-426c-8478-536b2070cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.688445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51640d4-0a90-4619-83c7-a063a61ed34f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.744125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1344/2668] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:42.752435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefc936-112d-43ed-b627-762da74f57ab None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.752758 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefc936-112d-43ed-b627-762da74f57ab None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.753017 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefc936-112d-43ed-b627-762da74f57ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.767577 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.768264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.768542 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.769396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.770252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.772286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-eaefc936-112d-43ed-b627-762da74f57ab None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:42.772849 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.773640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-eaefc936-112d-43ed-b627-762da74f57ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.784352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1345/2669] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:42.788393 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1325/2670] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:42.788711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefc936-112d-43ed-b627-762da74f57ab None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.789835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.800259 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefc936-112d-43ed-b627-762da74f57ab None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.800448 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22082e0-548f-412a-844c-7d61720229d4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.805445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.806112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.806388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.806661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.806778 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.807441 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.807550 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.808184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.808477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.808971 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.809237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:42.809753 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.811729 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.822914 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1326/2671] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:42.827230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1346/2672] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:20:42.827507 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.832028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.838842 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e2cc-b1d5-46f2-99be-af4fe4ea79a6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.843106 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505560b8-141f-4492-bf56-4849a1cfbed7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:42.843915 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.845412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.845817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.846132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] PATH_INFO: `/v3/projects/f901e64bf5d8440e886b97f746573d95/users/46249e04557249bfb9a9b68313131d4c/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.848910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdead89-87b1-41da-bdfd-c12a5fe8f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:42.849664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdead89-87b1-41da-bdfd-c12a5fe8f5bc admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.849982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdead89-87b1-41da-bdfd-c12a5fe8f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.850612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdead89-87b1-41da-bdfd-c12a5fe8f5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.851429 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdead89-87b1-41da-bdfd-c12a5fe8f5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.853486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdead89-87b1-41da-bdfd-c12a5fe8f5bc admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.863970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f901e64bf5d8440e886b97f746573d95, user_id=46249e04557249bfb9a9b68313131d4c, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:42.867482 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:42.882412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1327/2673] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:42 2022] PUT /identity/v3/projects/f901e64bf5d8440e886b97f746573d95/users/46249e04557249bfb9a9b68313131d4c/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:20:42.883223 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1347/2674] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:20:42.890108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:42.892954 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:42.893271 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:42.893541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:42.922389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] MFA Rules not processed for user `46249e04557249bfb9a9b68313131d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:42.923829 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:42.952502 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:42.963689 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78264e-d2a5-4b75-a6d6-29ef6c129566 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.007300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1348/2675] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:43.008168 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.009461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.032667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.033378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.033661 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.033938 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.034627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.036534 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.060468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1328/2676] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:42 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:43.060468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.060948 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.062015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:43.079434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eb773-32aa-4a12-b995-97064213d770 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.084055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.084925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.085230 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.085500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] PATH_INFO: `/v3/projects/2e69e489c0874765aab8316f992688e2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.086676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e69e489c0874765aab8316f992688e2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.088583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.137321 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e69e489c0874765aab8316f992688e2', 'request_id': 'req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.140639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba158a-6ccd-4510-8f05-058a1ad9e4f3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.143734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e69e489c0874765aab8316f992688e2', 'request_id': 'req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.149447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e69e489c0874765aab8316f992688e2', 'request_id': 'req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.163055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] The token cache is being invalidate because project 2e69e489c0874765aab8316f992688e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:43.163466 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.164438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.166594 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1349/2677] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:43 2022] DELETE /identity/v3/projects/2e69e489c0874765aab8316f992688e2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:43.167904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.170483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.217005 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.217833 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.218113 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.218388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.220234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.236121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91e2d-fa75-4e76-8a01-3e813bb2e266 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.240958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.241664 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.241917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.242213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.242918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.244534 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.245896 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:43.256033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1350/2678] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:43.256869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.259335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.272263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09578350-e195-457d-8a56-eacf592d2369 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.277384 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.279439 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.279703 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.280248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] PATH_INFO: `/v3/users/b989c61d94c34fa9acc2d5373b603e9e` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.280862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.281579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b989c61d94c34fa9acc2d5373b603e9e)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.282526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.283623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.291426 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1329/2679] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:43.292211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.303516 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff68c2-94e5-4e1e-8639-453cb57cc165 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.308475 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.309178 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.309559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.309878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.311209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.313928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.324895 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1330/2680] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:43.329262 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.339538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860dd0-4798-42b8-8987-0c3d95dc54e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.344222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899099c7-c7df-4f5a-ab41-51eee701a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.345277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899099c7-c7df-4f5a-ab41-51eee701a8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.345569 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899099c7-c7df-4f5a-ab41-51eee701a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.345917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b989c61d94c34fa9acc2d5373b603e9e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.346044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899099c7-c7df-4f5a-ab41-51eee701a8aa admin admin] PATH_INFO: `/v3/projects/883812495f3940a3bdeff8d4c08a29c3/users/5279c7bf349746b4a4622f72945662ea/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.346715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b989c61d94c34fa9acc2d5373b603e9e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.347348 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b989c61d94c34fa9acc2d5373b603e9e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.361539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b989c61d94c34fa9acc2d5373b603e9e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.362854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:43.363555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899099c7-c7df-4f5a-ab41-51eee701a8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=883812495f3940a3bdeff8d4c08a29c3, user_id=5279c7bf349746b4a4622f72945662ea, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.366614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899099c7-c7df-4f5a-ab41-51eee701a8aa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.371563 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b989c61d94c34fa9acc2d5373b603e9e'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.381456 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1331/2681] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:43 2022] PUT /identity/v3/projects/883812495f3940a3bdeff8d4c08a29c3/users/5279c7bf349746b4a4622f72945662ea/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:43.386909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.387198 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.387505 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.392988 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1351/2682] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:43 2022] DELETE /identity/v3/users/b989c61d94c34fa9acc2d5373b603e9e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:43.409409 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:43.410669 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.480165 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1332/2683] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:43.489909 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.491601 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.564370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a709f-5625-4a35-9966-384ef2a9a542 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.570609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c0666-cbe3-4e37-a593-abc712de31ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.572089 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c0666-cbe3-4e37-a593-abc712de31ce service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.572517 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c0666-cbe3-4e37-a593-abc712de31ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.573001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c0666-cbe3-4e37-a593-abc712de31ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.586672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c0666-cbe3-4e37-a593-abc712de31ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.588256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c0666-cbe3-4e37-a593-abc712de31ce service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.600376 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1352/2684] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:43.732055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:43.733819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.752908 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa36239-7e91-4b63-a2ff-c751fdef35e7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:43.760205 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:43.761274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.761705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.761981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] PATH_INFO: `/v3/projects/bc1d1d7f144849e2b253f23948d52933` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.763279 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1d1d7f144849e2b253f23948d52933)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:43.766118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:43.821504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1d1d7f144849e2b253f23948d52933', 'request_id': 'req-7edd97b3-f69e-48ec-8300-a9c450f859ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.830533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1d1d7f144849e2b253f23948d52933', 'request_id': 'req-7edd97b3-f69e-48ec-8300-a9c450f859ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.837389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1d1d7f144849e2b253f23948d52933', 'request_id': 'req-7edd97b3-f69e-48ec-8300-a9c450f859ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.850885 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] The token cache is being invalidate because project bc1d1d7f144849e2b253f23948d52933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:43.851280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.852129 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7edd97b3-f69e-48ec-8300-a9c450f859ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:43.855493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1333/2685] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:43 2022] DELETE /identity/v3/projects/bc1d1d7f144849e2b253f23948d52933 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:43.860507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46e6f-418f-4f8f-a1a0-81c7fd77c7d0 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.860812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46e6f-418f-4f8f-a1a0-81c7fd77c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.861135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46e6f-418f-4f8f-a1a0-81c7fd77c7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.880496 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-4be46e6f-418f-4f8f-a1a0-81c7fd77c7d0 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:43.881889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-4be46e6f-418f-4f8f-a1a0-81c7fd77c7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.946341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1353/2686] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:43.954101 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eedf91-562c-4f6c-be4d-836fd9722a3c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:43.954525 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eedf91-562c-4f6c-be4d-836fd9722a3c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:43.954786 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eedf91-562c-4f6c-be4d-836fd9722a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:43.974680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-64eedf91-562c-4f6c-be4d-836fd9722a3c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:43.976269 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-64eedf91-562c-4f6c-be4d-836fd9722a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:43.985708 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1334/2687] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:43.993991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46e6f-418f-4f8f-a1a0-81c7fd77c7d0 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:44.004719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46e6f-418f-4f8f-a1a0-81c7fd77c7d0 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:44.009933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:44.010609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.010887 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.011161 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.011926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:44.012255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:44.014177 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:44.031333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1354/2688] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:44.034007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eedf91-562c-4f6c-be4d-836fd9722a3c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:44.046079 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eedf91-562c-4f6c-be4d-836fd9722a3c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:44.050931 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:44.051679 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.052038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.052399 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.053673 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:44.056203 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:44.061030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:44.094325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1335/2689] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:44 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:44.095555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:44.105940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05c546-e1f6-48cb-8eaa-efb4a2437e96 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:44.165630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:44.166470 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.166749 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.167031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.167751 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:44.170432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:44.192933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1355/2690] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:44 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:44.197098 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:44.207899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546795e4-a143-4f9d-8f76-6fd747af8c03 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:44.212587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:44.213508 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.213707 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.214042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.214834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:44.216557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:44.227481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1336/2691] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:44.231582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:44.237953 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:44.243700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd33ef-1c91-4e46-8d09-192c72625392 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:44.250452 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41048a26-ff6a-4091-b6c0-f643a445ab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:44.251247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048a26-ff6a-4091-b6c0-f643a445ab70 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.251832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048a26-ff6a-4091-b6c0-f643a445ab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.252406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048a26-ff6a-4091-b6c0-f643a445ab70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.253361 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41048a26-ff6a-4091-b6c0-f643a445ab70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:44.255279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41048a26-ff6a-4091-b6c0-f643a445ab70 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:44.269520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1356/2692] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:44.270741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:44.282424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64372531-be93-49ee-89f0-f7541ab68f5b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:44.288726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:44.289893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.290040 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.290214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] PATH_INFO: `/v3/projects/3f78fe1ecaca42d2ae09999dcb3c8d7c/users/1aff23c6c4f6473eba8c6bc659345ba9/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.308568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f78fe1ecaca42d2ae09999dcb3c8d7c, user_id=1aff23c6c4f6473eba8c6bc659345ba9, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:44.311912 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:44.328665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1337/2693] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:44 2022] PUT /identity/v3/projects/3f78fe1ecaca42d2ae09999dcb3c8d7c/users/1aff23c6c4f6473eba8c6bc659345ba9/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:44.334302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e94b4-7b35-4c08-a3de-bec56d6ae149 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.334917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e94b4-7b35-4c08-a3de-bec56d6ae149 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.335339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e94b4-7b35-4c08-a3de-bec56d6ae149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.358234 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-e94e94b4-7b35-4c08-a3de-bec56d6ae149 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:44.359493 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e94e94b4-7b35-4c08-a3de-bec56d6ae149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:44.447115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1357/2694] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:44.454197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:44.456611 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:44.520930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a191b2-2f01-4c0e-a3e2-7903e7c23006 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:44.524517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:44.525210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:44.525518 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:44.525791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:44.538633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:44.540132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:44.551398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1338/2695] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:45.599895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:45.600509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:45.600509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:45.635374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] MFA Rules not processed for user `1aff23c6c4f6473eba8c6bc659345ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:45.636538 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:45.729349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1358/2696] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:45.740167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:45.742336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:45.815297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03d0ea-47cf-49a7-870b-e68488d478df service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:45.819352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ce149-e3ca-4529-84d3-aa1a27fff996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:45.820252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce149-e3ca-4529-84d3-aa1a27fff996 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:45.820711 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce149-e3ca-4529-84d3-aa1a27fff996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:45.821239 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce149-e3ca-4529-84d3-aa1a27fff996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:45.830862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ce149-e3ca-4529-84d3-aa1a27fff996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:45.832353 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ce149-e3ca-4529-84d3-aa1a27fff996 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:45.867763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1339/2697] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:46.178288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:46.180342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:46.202644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dd6a8-e135-455a-84b8-896cabb2cbe7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:46.228378 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:46.231522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:46.233209 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:46.233583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] PATH_INFO: `/v3/users/46249e04557249bfb9a9b68313131d4c` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:46.235932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46249e04557249bfb9a9b68313131d4c)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:46.238514 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:46.314140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46249e04557249bfb9a9b68313131d4c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:46.314850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46249e04557249bfb9a9b68313131d4c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:46.315555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46249e04557249bfb9a9b68313131d4c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:46.340283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46249e04557249bfb9a9b68313131d4c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:46.352720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46249e04557249bfb9a9b68313131d4c'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:46.356162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:46.378515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1359/2698] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:46 2022] DELETE /identity/v3/users/46249e04557249bfb9a9b68313131d4c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:46.383248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:46.383541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:46.383819 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:46.409545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:46.411394 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:46.481117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:46.502071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1340/2699] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:46.515511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:46.518807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:46.638004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c258f5-d82c-4050-8994-1094bfe0f08e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:46.645424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243ed7c-4a0e-40f6-9136-01d5fd0507f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:46.646370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243ed7c-4a0e-40f6-9136-01d5fd0507f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:46.646881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243ed7c-4a0e-40f6-9136-01d5fd0507f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:46.647366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243ed7c-4a0e-40f6-9136-01d5fd0507f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:46.673509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243ed7c-4a0e-40f6-9136-01d5fd0507f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:46.677090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243ed7c-4a0e-40f6-9136-01d5fd0507f8 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:46.708705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1360/2700] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:47.156415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:47.158064 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:47.171635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad1ac1-c77c-4ad3-9582-edc08a7e991b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:47.187676 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:47.189887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.190223 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.190549 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] PATH_INFO: `/v3/projects/f901e64bf5d8440e886b97f746573d95` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.192863 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f901e64bf5d8440e886b97f746573d95)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:47.195693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:47.284927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f901e64bf5d8440e886b97f746573d95', 'request_id': 'req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:47.296946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f901e64bf5d8440e886b97f746573d95', 'request_id': 'req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:47.322817 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f901e64bf5d8440e886b97f746573d95', 'request_id': 'req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:47.341028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] The token cache is being invalidate because project f901e64bf5d8440e886b97f746573d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:47.341467 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:47.342597 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-7ecd68c9-9b1d-4f18-b51e-d5198b0fc777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:47.347881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1341/2701] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:47 2022] DELETE /identity/v3/projects/f901e64bf5d8440e886b97f746573d95 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:47.355055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3acd9-9689-4542-b819-68038b91bcae None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.355499 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3acd9-9689-4542-b819-68038b91bcae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.355870 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3acd9-9689-4542-b819-68038b91bcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.376841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-24f3acd9-9689-4542-b819-68038b91bcae None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:47.378433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-24f3acd9-9689-4542-b819-68038b91bcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:47.385978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f456f49-f261-42df-bc30-0136d7b7566b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.386300 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f456f49-f261-42df-bc30-0136d7b7566b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.386627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f456f49-f261-42df-bc30-0136d7b7566b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.428911 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6f456f49-f261-42df-bc30-0136d7b7566b None None] MFA Rules not processed for user `1aff23c6c4f6473eba8c6bc659345ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:47.429712 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6f456f49-f261-42df-bc30-0136d7b7566b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:47.467365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1361/2702] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:47.477024 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2249781-fbde-4383-9105-82d877b9cf39 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.481037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2249781-fbde-4383-9105-82d877b9cf39 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.481424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2249781-fbde-4383-9105-82d877b9cf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.508443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-f2249781-fbde-4383-9105-82d877b9cf39 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:47.509708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f2249781-fbde-4383-9105-82d877b9cf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:47.530767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1362/2703] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:47.531777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2249781-fbde-4383-9105-82d877b9cf39 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:47.534481 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1342/2704] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:47.574533 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2249781-fbde-4383-9105-82d877b9cf39 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:47.594280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:47.594985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.595272 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.595603 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.596351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:47.596658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:47.598527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:47.631187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1363/2705] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:47.633701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f456f49-f261-42df-bc30-0136d7b7566b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:47.649871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f456f49-f261-42df-bc30-0136d7b7566b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:47.657227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:47.658547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.659013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.659468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.660837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:47.663630 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:47.722026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1343/2706] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:47 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:47.722300 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:47.734546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19321517-579d-439c-af36-2dd0c397072a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:47.851919 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:47.853650 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.853949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.854379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.857237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:47.861427 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:47.891400 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1364/2707] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:47 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:47.893222 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:47.908284 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9aaec-1067-4e67-bc9a-43c508daf7bf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:47.914250 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:47.914913 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.915195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.915468 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.921412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:47.924699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:47.943587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1344/2708] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:47.944532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:47.960183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d631-cfc1-4ced-9a8c-239e499406f4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:47.966727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c4450-d520-418d-ab60-623e2c5b4fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:47.967516 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c4450-d520-418d-ab60-623e2c5b4fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:47.968004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c4450-d520-418d-ab60-623e2c5b4fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:47.968345 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c4450-d520-418d-ab60-623e2c5b4fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:47.969953 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c4450-d520-418d-ab60-623e2c5b4fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:47.971744 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c4450-d520-418d-ab60-623e2c5b4fcb admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:47.973454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:47.993888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1365/2709] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:20:47.995082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.014006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b29deb-b05a-422e-ac53-6bc1f50ba8ab admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.031531 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:48.032423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.032699 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.032969 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] PATH_INFO: `/v3/projects/8e97b84ef5e54dd080c38caf15c51dc3/users/eecc5704ec904da1b6565e368e79ba4e/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.066741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e97b84ef5e54dd080c38caf15c51dc3, user_id=eecc5704ec904da1b6565e368e79ba4e, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:48.070930 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:48.102240 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1345/2710] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:47 2022] PUT /identity/v3/projects/8e97b84ef5e54dd080c38caf15c51dc3/users/eecc5704ec904da1b6565e368e79ba4e/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:48.113022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e665-db72-4e71-b91e-9c2e473d55b7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.113588 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e665-db72-4e71-b91e-9c2e473d55b7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.113946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e665-db72-4e71-b91e-9c2e473d55b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.147920 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-0b76e665-db72-4e71-b91e-9c2e473d55b7 None None] MFA Rules not processed for user `eecc5704ec904da1b6565e368e79ba4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:48.149622 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-0b76e665-db72-4e71-b91e-9c2e473d55b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:48.249454 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1366/2711] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:48.254072 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.301713 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:48.350677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53fa60-3100-4c49-ba4a-9a50afbece94 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.357845 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:48.358552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.358837 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.359112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.359820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:48.361828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:48.411880 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1346/2712] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:48 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:48.413491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e665-db72-4e71-b91e-9c2e473d55b7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.437009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e665-db72-4e71-b91e-9c2e473d55b7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.550663 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:48.551282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.551568 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.551836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.552527 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:48.554476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:48.588204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1367/2713] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:48 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:48.589415 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.604719 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53e8bc-5bcc-4112-820c-e0e5e29c777e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.618681 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc8e26-30ba-41f1-a910-c3a1c5e7685a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:48.619875 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc8e26-30ba-41f1-a910-c3a1c5e7685a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.620336 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc8e26-30ba-41f1-a910-c3a1c5e7685a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.620622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc8e26-30ba-41f1-a910-c3a1c5e7685a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.621424 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc8e26-30ba-41f1-a910-c3a1c5e7685a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:48.623349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc8e26-30ba-41f1-a910-c3a1c5e7685a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:48.654771 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1347/2714] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:48.658657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.672487 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94ff2e-c137-4d80-b132-996f43e92143 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.678178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:48.679207 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.679557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.679906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] PATH_INFO: `/v3/projects/ca5271ada7f84356bab6bd00f063ed90/users/a8e3645beceb4316af205dfb236e39cc/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.706075 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5271ada7f84356bab6bd00f063ed90, user_id=a8e3645beceb4316af205dfb236e39cc, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:48.708910 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:48.732136 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1368/2715] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:48 2022] PUT /identity/v3/projects/ca5271ada7f84356bab6bd00f063ed90/users/a8e3645beceb4316af205dfb236e39cc/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:48.737348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.737659 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.737937 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.738835 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.786615 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] MFA Rules not processed for user `5279c7bf349746b4a4622f72945662ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:48.788154 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:48.809986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:48.826285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b983e3-7e30-432f-96dd-27f6f6e95af7 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.834438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:48.835142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.835418 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.835684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.836417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:48.838056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:48.851931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1369/2716] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 07:20:48.855963 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.869461 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0177-f9f3-40a1-ae71-c191bfd0bd1e admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.880555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d4f18-0cf5-451e-b572-fd5dbba7f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:48.881575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d4f18-0cf5-451e-b572-fd5dbba7f624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.881856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d4f18-0cf5-451e-b572-fd5dbba7f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.882135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d4f18-0cf5-451e-b572-fd5dbba7f624 admin admin] PATH_INFO: `/v3/projects/ca5271ada7f84356bab6bd00f063ed90/users/a8e3645beceb4316af205dfb236e39cc/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.884577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d4f18-0cf5-451e-b572-fd5dbba7f624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5271ada7f84356bab6bd00f063ed90, user_id=a8e3645beceb4316af205dfb236e39cc, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:48.887671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d4f18-0cf5-451e-b572-fd5dbba7f624 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:48.902075 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1348/2717] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:48.903418 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:48.904802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:48.905143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1370/2718] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:48 2022] PUT /identity/v3/projects/ca5271ada7f84356bab6bd00f063ed90/users/a8e3645beceb4316af205dfb236e39cc/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:48.919750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:48.920368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:48.920864 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:48.927684 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f851d1-4532-4a11-b146-e422f43dc09b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:48.959057 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] MFA Rules not processed for user `a8e3645beceb4316af205dfb236e39cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:48.961066 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:49.005038 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:49.005677 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:49.006028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:49.006465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:49.007428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:49.009700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:49.042528 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1349/2719] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:48 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:49.046654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:49.057172 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1371/2720] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:49.071518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:49.073225 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:49.075254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:49.167928 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4684-a13c-40a8-9112-b546aae8b988 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:49.181233 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:49.181963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:49.184266 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:49.184634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:49.186225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:49.190946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:49.197810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f4be1-f1cd-4d0c-9d91-89967e4def13 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:49.239889 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1350/2721] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:49 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:20:49.241006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:49.259691 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9c4a8-9f9f-44fb-abe6-2ab177cdd3b7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:49.273264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:49.274587 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:49.275288 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:49.275763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:49.276878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:49.279757 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:49.297194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1351/2722] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:49.298413 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:49.307985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:49.308736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:49.309115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:49.309593 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:49.318509 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0defbac-159a-44a2-a626-e664008fe5fe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:49.327083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:49.328378 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:49.328603 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:49.328878 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] PATH_INFO: `/v3/projects/31ed8ccb90b1421bb77d691f395b43d5/users/6b6c519eb06a4a4cb62f38624f7f6228/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:49.354803 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ed8ccb90b1421bb77d691f395b43d5, user_id=6b6c519eb06a4a4cb62f38624f7f6228, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:49.359398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:49.364635 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:49.390562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1352/2723] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:49 2022] PUT /identity/v3/projects/31ed8ccb90b1421bb77d691f395b43d5/users/6b6c519eb06a4a4cb62f38624f7f6228/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:49.394030 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:49.434997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:49.436450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:49.447323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1372/2724] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:49.537555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e315ff7-090e-4905-953b-d4252d5c4ea6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:49.550359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:49.551150 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:49.551432 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:49.551702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:49.552497 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:49.554067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:49.567555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1353/2725] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:49.572782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:49.594997 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebaabd-5ab2-41f6-bb6f-a12a4ee3fd99 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:49.601070 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21ca3a-3086-42f7-978a-63c98b383ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:49.602576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ca3a-3086-42f7-978a-63c98b383ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:49.602576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ca3a-3086-42f7-978a-63c98b383ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:49.602790 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ca3a-3086-42f7-978a-63c98b383ae1 admin admin] PATH_INFO: `/v3/projects/31ed8ccb90b1421bb77d691f395b43d5/users/6b6c519eb06a4a4cb62f38624f7f6228/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:49.604657 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21ca3a-3086-42f7-978a-63c98b383ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ed8ccb90b1421bb77d691f395b43d5, user_id=6b6c519eb06a4a4cb62f38624f7f6228, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:49.608249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21ca3a-3086-42f7-978a-63c98b383ae1 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:49.620623 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1373/2726] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:49 2022] PUT /identity/v3/projects/31ed8ccb90b1421bb77d691f395b43d5/users/6b6c519eb06a4a4cb62f38624f7f6228/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:50.034445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:50.044590 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549d40-73e2-40c2-a52f-19ece9564215 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:50.110545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:50.111454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:50.111727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:50.111998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] PATH_INFO: `/v3/users/eecc5704ec904da1b6565e368e79ba4e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:50.113869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eecc5704ec904da1b6565e368e79ba4e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:50.116021 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:50.123812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:50.173274 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eecc5704ec904da1b6565e368e79ba4e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:50.174006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eecc5704ec904da1b6565e368e79ba4e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:50.189010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eecc5704ec904da1b6565e368e79ba4e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:50.189851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eecc5704ec904da1b6565e368e79ba4e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:50.201607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eecc5704ec904da1b6565e368e79ba4e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:50.219945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1354/2727] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:50 2022] DELETE /identity/v3/users/eecc5704ec904da1b6565e368e79ba4e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:50.224914 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:50.225616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:50.225616 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:50.244604 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:50.245854 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:50.333137 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1374/2728] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:50.344732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:50.346701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:50.415231 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb8271-e94a-402f-ac84-1b6c32d264c9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:50.419055 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:50.420028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:50.420305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:50.420574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:50.430319 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:50.431766 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:50.442489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1355/2729] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:50.954705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:50.955908 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:50.977923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481914-ddb3-43e1-9cef-ee09be6ea98e None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:50.984170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:50.985192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:50.985471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:50.985882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] PATH_INFO: `/v3/projects/8e97b84ef5e54dd080c38caf15c51dc3` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:50.987332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e97b84ef5e54dd080c38caf15c51dc3)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:50.989980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:51.053823 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e97b84ef5e54dd080c38caf15c51dc3', 'request_id': 'req-af937176-44a0-459b-a899-9d7142ff872f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.062899 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e97b84ef5e54dd080c38caf15c51dc3', 'request_id': 'req-af937176-44a0-459b-a899-9d7142ff872f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.071392 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e97b84ef5e54dd080c38caf15c51dc3', 'request_id': 'req-af937176-44a0-459b-a899-9d7142ff872f'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.093016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] The token cache is being invalidate because project 8e97b84ef5e54dd080c38caf15c51dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:51.093775 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.094679 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.101852 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1375/2730] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:50 2022] DELETE /identity/v3/projects/8e97b84ef5e54dd080c38caf15c51dc3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:51.103477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:51.113984 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c28a-158b-4a93-b25b-26da52933db3 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:51.185706 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:51.186705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:51.187435 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:51.187805 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] PATH_INFO: `/v3/users/a8e3645beceb4316af205dfb236e39cc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:51.189254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e3645beceb4316af205dfb236e39cc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:51.191890 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:51.241310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:51.272589 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e3645beceb4316af205dfb236e39cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.273493 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e3645beceb4316af205dfb236e39cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.300596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e3645beceb4316af205dfb236e39cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.314207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e3645beceb4316af205dfb236e39cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.326872 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c892f38f-fb83-4244-b991-17eae0c19f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e3645beceb4316af205dfb236e39cc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.344690 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1356/2731] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:51 2022] DELETE /identity/v3/users/a8e3645beceb4316af205dfb236e39cc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:51.689346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:51.691787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:51.711747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af937176-44a0-459b-a899-9d7142ff872f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:51.753140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:51.806522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:51.807809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:51.808306 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:51.808931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] PATH_INFO: `/v3/projects/ca5271ada7f84356bab6bd00f063ed90` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:51.811140 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5271ada7f84356bab6bd00f063ed90)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:51.815260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:51.902459 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5271ada7f84356bab6bd00f063ed90', 'request_id': 'req-d9840a01-31e6-4214-9fe1-f7576eefd158'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.917451 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5271ada7f84356bab6bd00f063ed90', 'request_id': 'req-d9840a01-31e6-4214-9fe1-f7576eefd158'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.925408 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5271ada7f84356bab6bd00f063ed90', 'request_id': 'req-d9840a01-31e6-4214-9fe1-f7576eefd158'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.950952 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] The token cache is being invalidate because project ca5271ada7f84356bab6bd00f063ed90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:20:51.951414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.962450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d9840a01-31e6-4214-9fe1-f7576eefd158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:51.964582 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1376/2732] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:20:51 2022] DELETE /identity/v3/projects/ca5271ada7f84356bab6bd00f063ed90 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:51.970372 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1c18c-9ef4-456c-a09d-a9ff11c4901e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:51.971466 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1c18c-9ef4-456c-a09d-a9ff11c4901e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:51.971986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1c18c-9ef4-456c-a09d-a9ff11c4901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:51.996318 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-20a1c18c-9ef4-456c-a09d-a9ff11c4901e None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:51.998351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-20a1c18c-9ef4-456c-a09d-a9ff11c4901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:52.071884 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1357/2733] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:52.073760 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb67089-b51f-4955-8225-889b68832a5c None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.074113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb67089-b51f-4955-8225-889b68832a5c None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.074315 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb67089-b51f-4955-8225-889b68832a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.095421 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-bdb67089-b51f-4955-8225-889b68832a5c None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:52.096648 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-bdb67089-b51f-4955-8225-889b68832a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:52.104707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1377/2734] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:52.109189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1c18c-9ef4-456c-a09d-a9ff11c4901e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:52.119006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1c18c-9ef4-456c-a09d-a9ff11c4901e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:52.123729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:52.124425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.125068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.125068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.125739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:20:52.126046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:52.127656 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:52.146535 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1358/2735] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:20:52.150857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb67089-b51f-4955-8225-889b68832a5c None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:52.168282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb67089-b51f-4955-8225-889b68832a5c None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:52.174171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:52.174936 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.175285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.175618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.176463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:52.178672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:52.217110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1378/2736] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:52 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:52.221213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:52.248010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f01a-8f96-4003-9b85-43600a3083fa admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:52.367886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:52.368797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.369620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.370220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.371067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:52.378028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:52.426744 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1359/2737] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:52 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:52.430770 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:52.462214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e63c52-7b6f-4c45-ac6c-eac1e6ec4cb6 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:52.485797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:52.486602 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.486945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.487287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.488111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:52.489866 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:52.502605 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1379/2738] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:52.511107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:52.529741 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70aed-f348-414e-8cfa-9137214039ff admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:52.537526 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c34f4-90f9-4e58-a8dd-cc959abd096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:52.538440 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c34f4-90f9-4e58-a8dd-cc959abd096a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.538816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c34f4-90f9-4e58-a8dd-cc959abd096a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.539341 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c34f4-90f9-4e58-a8dd-cc959abd096a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.540221 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c34f4-90f9-4e58-a8dd-cc959abd096a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:52.554009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c34f4-90f9-4e58-a8dd-cc959abd096a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:52.594985 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1360/2739] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:52.597671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:52.615599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8152-b366-4bc2-bcc8-3e0ca86ed532 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:52.636278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:52.639424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.639859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.640233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] PATH_INFO: `/v3/projects/2a86c23bfc2d4985961daa6e934f8d26/users/8472d375b4464807864afb2c67653b85/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.737279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a86c23bfc2d4985961daa6e934f8d26, user_id=8472d375b4464807864afb2c67653b85, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:52.743630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:52.783434 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1380/2740] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:52 2022] PUT /identity/v3/projects/2a86c23bfc2d4985961daa6e934f8d26/users/8472d375b4464807864afb2c67653b85/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:52.794752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942123bc-7526-44bd-a8fd-fef2080a7270 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:52.795416 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942123bc-7526-44bd-a8fd-fef2080a7270 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:52.795910 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942123bc-7526-44bd-a8fd-fef2080a7270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:52.834024 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:52.839090 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-942123bc-7526-44bd-a8fd-fef2080a7270 None None] MFA Rules not processed for user `8472d375b4464807864afb2c67653b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:52.841501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-942123bc-7526-44bd-a8fd-fef2080a7270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:52.947644 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1361/2741] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:20:52.952856 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:53.061966 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cf4b1-7610-42d7-bbd9-16cdd64797c2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:53.071767 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:53.072475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:53.072812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:53.072994 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:53.073740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:53.075918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:53.081615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:53.115727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1381/2742] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:20:52 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:53.119573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942123bc-7526-44bd-a8fd-fef2080a7270 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:53.144215 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942123bc-7526-44bd-a8fd-fef2080a7270 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:53.240602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:53.242457 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:53.243820 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:53.245091 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:53.246581 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:53.250131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:53.310757 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:53.312326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1362/2743] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:20:53 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:20:53.329337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865503-9b74-4b7c-93a6-fe258e2e9f2b admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:53.349098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:53.351727 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:53.352014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:53.352287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:53.353119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:53.361382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:53.383940 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1382/2744] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:53.387926 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:53.407242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56ae3-ccca-49a9-9053-2c318e782cf3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:53.419170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:53.420267 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:53.420555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:53.420834 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] PATH_INFO: `/v3/projects/a6a256fb3c7d43f082bea17f359806ed/users/013c7326c22b424c84360ccd971cc412/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:53.447174 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a256fb3c7d43f082bea17f359806ed, user_id=013c7326c22b424c84360ccd971cc412, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:53.450417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:53.478791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1363/2745] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:53 2022] PUT /identity/v3/projects/a6a256fb3c7d43f082bea17f359806ed/users/013c7326c22b424c84360ccd971cc412/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:53.482737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:53.500812 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa55c-6438-412d-9734-e70ae444d7af admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:53.679841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7bf7a-c1d8-4cce-aee2-8db24619d1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:53.680587 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7bf7a-c1d8-4cce-aee2-8db24619d1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:53.680874 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7bf7a-c1d8-4cce-aee2-8db24619d1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:53.681249 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7bf7a-c1d8-4cce-aee2-8db24619d1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:53.683513 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7bf7a-c1d8-4cce-aee2-8db24619d1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:53.695946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7bf7a-c1d8-4cce-aee2-8db24619d1c6 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:53.734561 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1383/2746] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:20:53.739627 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:53.754835 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5306c32-598e-4c05-83b7-7616594af4f0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:53.770693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:53.771619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:53.771899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:53.772170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] PATH_INFO: `/v3/projects/a6a256fb3c7d43f082bea17f359806ed/users/013c7326c22b424c84360ccd971cc412/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:53.775486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a256fb3c7d43f082bea17f359806ed, user_id=013c7326c22b424c84360ccd971cc412, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:53.779851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:53.821412 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1364/2747] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:20:53 2022] PUT /identity/v3/projects/a6a256fb3c7d43f082bea17f359806ed/users/013c7326c22b424c84360ccd971cc412/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:53.832630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8989bf-1aa8-4359-b824-ff2da81b7eae None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:53.833181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8989bf-1aa8-4359-b824-ff2da81b7eae None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:53.833181 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8989bf-1aa8-4359-b824-ff2da81b7eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:53.917630 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5f8989bf-1aa8-4359-b824-ff2da81b7eae None None] MFA Rules not processed for user `013c7326c22b424c84360ccd971cc412`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:53.924507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5f8989bf-1aa8-4359-b824-ff2da81b7eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:54.131406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1384/2748] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:54.144152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:54.146881 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:54.271749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbfc6e-1531-4880-bce6-bb91f17511f6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:54.276391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01942c51-9da1-4245-84fe-3d6dc8025999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:54.277189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01942c51-9da1-4245-84fe-3d6dc8025999 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:54.277970 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01942c51-9da1-4245-84fe-3d6dc8025999 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:54.278901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01942c51-9da1-4245-84fe-3d6dc8025999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:54.292981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01942c51-9da1-4245-84fe-3d6dc8025999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:54.306877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01942c51-9da1-4245-84fe-3d6dc8025999 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:54.316877 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:54.353660 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1365/2749] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:54.507982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8989bf-1aa8-4359-b824-ff2da81b7eae None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:54.529155 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8989bf-1aa8-4359-b824-ff2da81b7eae None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:54.534589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:54.535468 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:54.535831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:54.536715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:54.604266 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:54.608264 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:54.609960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:54.622564 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1385/2750] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:55.880156 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778c873-b312-45ea-ac2b-6ecb349171c5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:55.880635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778c873-b312-45ea-ac2b-6ecb349171c5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:55.880768 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778c873-b312-45ea-ac2b-6ecb349171c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:55.907168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2778c873-b312-45ea-ac2b-6ecb349171c5 None None] MFA Rules not processed for user `6b6c519eb06a4a4cb62f38624f7f6228`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:55.908302 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2778c873-b312-45ea-ac2b-6ecb349171c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:55.979067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1366/2751] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:20:55.994228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:55.995960 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:56.080112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b97611-3900-4deb-a545-9d020437b093 service cinder] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:56.086759 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e410b-abe7-4d4d-858a-6828d3d9dda9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:56.087391 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e410b-abe7-4d4d-858a-6828d3d9dda9 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:56.087700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e410b-abe7-4d4d-858a-6828d3d9dda9 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:56.087978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e410b-abe7-4d4d-858a-6828d3d9dda9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:56.089986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-363e410b-abe7-4d4d-858a-6828d3d9dda9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:20:56.150058 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e410b-abe7-4d4d-858a-6828d3d9dda9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:56.151653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e410b-abe7-4d4d-858a-6828d3d9dda9 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:56.162520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1386/2752] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:20:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:20:59.675574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778c873-b312-45ea-ac2b-6ecb349171c5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:20:59.789573 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778c873-b312-45ea-ac2b-6ecb349171c5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:20:59.794556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:20:59.795446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:59.795727 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:59.796007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] PATH_INFO: `/v3/users/8472d375b4464807864afb2c67653b85` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:59.797367 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8472d375b4464807864afb2c67653b85)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:20:59.799348 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:20:59.846346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:20:59.849538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8472d375b4464807864afb2c67653b85'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:59.850655 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8472d375b4464807864afb2c67653b85'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:59.863726 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8472d375b4464807864afb2c67653b85'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:59.864377 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8472d375b4464807864afb2c67653b85'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:59.871173 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8472d375b4464807864afb2c67653b85'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:20:59.885732 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1367/2753] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:20:59 2022] DELETE /identity/v3/users/8472d375b4464807864afb2c67653b85 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:20:59.890779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:20:59.891274 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:20:59.891409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:20:59.942129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:20:59.943312 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:00.014353 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1387/2754] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:21:00.025022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:00.026453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:00.102093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268b1e6-885a-4f9e-8747-d87d8822d9c9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:00.106346 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:00.107089 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:00.107376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:00.107653 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:00.117252 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:00.118718 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:00.129039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1368/2755] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:00.401800 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:00.403797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:00.418037 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d892ec6-4d44-4e3c-bf37-2b1b76766039 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:00.423807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:00.424687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:00.424959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:00.425257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] PATH_INFO: `/v3/projects/2a86c23bfc2d4985961daa6e934f8d26` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:00.426460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a86c23bfc2d4985961daa6e934f8d26)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:00.428373 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:00.430238 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=100 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:00.479113 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a86c23bfc2d4985961daa6e934f8d26', 'request_id': 'req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.485398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a86c23bfc2d4985961daa6e934f8d26', 'request_id': 'req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.491539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a86c23bfc2d4985961daa6e934f8d26', 'request_id': 'req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.506001 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] The token cache is being invalidate because project 2a86c23bfc2d4985961daa6e934f8d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:21:00.506275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.506925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.508529 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1388/2756] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:21:00 2022] DELETE /identity/v3/projects/2a86c23bfc2d4985961daa6e934f8d26 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:00.512844 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:00.522897 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91477-094d-448c-90c5-f76746a29fe9 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:00.579701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:00.580568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:00.580847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:00.581155 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] PATH_INFO: `/v3/users/013c7326c22b424c84360ccd971cc412` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:00.582371 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] RBAC: Authorizing `identity:delete_user(user_id=013c7326c22b424c84360ccd971cc412)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:00.584277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:00.633776 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013c7326c22b424c84360ccd971cc412'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.634417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013c7326c22b424c84360ccd971cc412'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.636281 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:00.649539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013c7326c22b424c84360ccd971cc412'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.650183 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013c7326c22b424c84360ccd971cc412'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.658080 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cafedc06-6487-4a8d-a4ed-47c28f82378a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013c7326c22b424c84360ccd971cc412'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:00.674737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1369/2757] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:21:00 2022] DELETE /identity/v3/users/013c7326c22b424c84360ccd971cc412 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:00.972836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:00.974476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:00.987797 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd9cb8-cec7-444d-bbb6-988dc84d75f8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.066178 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.067062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.067340 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.067612 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] PATH_INFO: `/v3/projects/a6a256fb3c7d43f082bea17f359806ed` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.068878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a256fb3c7d43f082bea17f359806ed)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.070861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.140330 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a256fb3c7d43f082bea17f359806ed', 'request_id': 'req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:01.146730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a256fb3c7d43f082bea17f359806ed', 'request_id': 'req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:01.154022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a256fb3c7d43f082bea17f359806ed', 'request_id': 'req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:01.168980 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] The token cache is being invalidate because project a6a256fb3c7d43f082bea17f359806ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:21:01.169289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:01.169933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-2d7769bb-9e38-44a3-85b7-28e00f2ca47d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:01.171709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1389/2758] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:21:00 2022] DELETE /identity/v3/projects/a6a256fb3c7d43f082bea17f359806ed => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:01.176551 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a8600-2dc9-45a5-b9d4-22d4ed85d748 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.177039 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a8600-2dc9-45a5-b9d4-22d4ed85d748 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.177186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a8600-2dc9-45a5-b9d4-22d4ed85d748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.229715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-013a8600-2dc9-45a5-b9d4-22d4ed85d748 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:01.230992 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-013a8600-2dc9-45a5-b9d4-22d4ed85d748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:01.292851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1370/2759] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:01.300839 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7cfc-97c7-45c6-92e2-cc2423728de9 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.301232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7cfc-97c7-45c6-92e2-cc2423728de9 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.301497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7cfc-97c7-45c6-92e2-cc2423728de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.320397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-9d9b7cfc-97c7-45c6-92e2-cc2423728de9 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:01.321618 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-9d9b7cfc-97c7-45c6-92e2-cc2423728de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:01.331174 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1390/2760] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:01.335517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a8600-2dc9-45a5-b9d4-22d4ed85d748 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.345761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a8600-2dc9-45a5-b9d4-22d4ed85d748 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.351193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.351853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.352130 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.352398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.353161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:21:01.353464 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.355083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.367145 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1371/2761] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:21:01.371478 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b7cfc-97c7-45c6-92e2-cc2423728de9 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.381333 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b7cfc-97c7-45c6-92e2-cc2423728de9 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.387447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.388265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.388546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.388820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.389545 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.391441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.402003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:01.421928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1391/2762] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:21:01 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:01.427780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.438496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74e674-f943-45ec-8c0f-754ca7129397 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.500868 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.501470 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.501752 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.502022 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.502708 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.504614 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.528472 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1372/2763] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:21:01 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:01.532734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.543303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e36dc6-99c7-4d90-8aa6-ad9a52098060 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.548520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.549219 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.549497 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.549780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.550505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.552118 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.562551 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1392/2764] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:01.566556 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.576946 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57957ae-5512-426b-af59-fd77f1c025eb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.582729 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a20487-9b91-4c33-88bb-981f68958174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.583408 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20487-9b91-4c33-88bb-981f68958174 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.583694 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20487-9b91-4c33-88bb-981f68958174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.583978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20487-9b91-4c33-88bb-981f68958174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.584692 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a20487-9b91-4c33-88bb-981f68958174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.586389 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a20487-9b91-4c33-88bb-981f68958174 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.602016 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1373/2765] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:01.606293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.616708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922b2f7-7df9-489d-a699-94d3b1044524 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.621488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.622390 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.622674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.623069 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] PATH_INFO: `/v3/projects/a8a70384eb524bb08ffc0f6e68861fa9/users/39bc03745c4142cda024730fd243c552/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.637978 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a70384eb524bb08ffc0f6e68861fa9, user_id=39bc03745c4142cda024730fd243c552, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.640709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.656876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1393/2766] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:01 2022] PUT /identity/v3/projects/a8a70384eb524bb08ffc0f6e68861fa9/users/39bc03745c4142cda024730fd243c552/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:01.667388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775be17-d6aa-4db1-8e56-227591f70ec7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.667678 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775be17-d6aa-4db1-8e56-227591f70ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.667951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775be17-d6aa-4db1-8e56-227591f70ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.686453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:01.696143 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a775be17-d6aa-4db1-8e56-227591f70ec7 None None] MFA Rules not processed for user `39bc03745c4142cda024730fd243c552`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:01.698023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a775be17-d6aa-4db1-8e56-227591f70ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:01.765167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1374/2767] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:01.769867 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.833660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db20784-3032-44c2-8ba1-803067b1fdd3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.839318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.839982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.840277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.840655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.841365 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.843278 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.870700 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1394/2768] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:21:01 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:01.874973 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775be17-d6aa-4db1-8e56-227591f70ec7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.885823 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775be17-d6aa-4db1-8e56-227591f70ec7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.943445 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.944126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.944443 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.944773 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.945504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.947478 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:01.973756 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1375/2769] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:21:01 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:01.977370 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:01.988479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e866e-322e-4c36-9133-ce7161068baa admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:01.993640 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:01.994396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:01.995073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:01.995073 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:01.995720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:01.997512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:02.009512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1395/2770] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:02.014163 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:02.024349 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8daf21f-1025-4187-85be-0ed9555e8c71 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:02.029538 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:02.030423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:02.030702 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:02.030967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] PATH_INFO: `/v3/projects/459e84940bef4257b1c62611aa3e8cb1/users/c115cd91f4b144c4a14ac7972a6b9d9d/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:02.049648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=459e84940bef4257b1c62611aa3e8cb1, user_id=c115cd91f4b144c4a14ac7972a6b9d9d, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:02.052423 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:02.068102 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1376/2771] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:02 2022] PUT /identity/v3/projects/459e84940bef4257b1c62611aa3e8cb1/users/c115cd91f4b144c4a14ac7972a6b9d9d/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:02.071944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:02.083138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffe78f-c337-4518-9879-af87f7a0408c admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:02.148192 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22073f44-ffe2-42e9-9ec5-4ee7ee7fadde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:02.148849 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22073f44-ffe2-42e9-9ec5-4ee7ee7fadde admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:02.149163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22073f44-ffe2-42e9-9ec5-4ee7ee7fadde admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:02.149443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22073f44-ffe2-42e9-9ec5-4ee7ee7fadde admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:02.150167 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22073f44-ffe2-42e9-9ec5-4ee7ee7fadde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:02.151799 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22073f44-ffe2-42e9-9ec5-4ee7ee7fadde admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:02.161926 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1396/2772] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:02.166265 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:02.176862 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e6db8-aaab-4a3d-903f-b3937db09ecd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:02.181812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:02.182734 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:02.183013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:02.183286 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] PATH_INFO: `/v3/projects/459e84940bef4257b1c62611aa3e8cb1/users/c115cd91f4b144c4a14ac7972a6b9d9d/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:02.185687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459e84940bef4257b1c62611aa3e8cb1, user_id=c115cd91f4b144c4a14ac7972a6b9d9d, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:02.188337 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:02.198761 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1377/2773] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:02 2022] PUT /identity/v3/projects/459e84940bef4257b1c62611aa3e8cb1/users/c115cd91f4b144c4a14ac7972a6b9d9d/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:02.210723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e7913-bc68-4ae4-a411-39105863d8c1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:02.211013 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e7913-bc68-4ae4-a411-39105863d8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:02.211290 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e7913-bc68-4ae4-a411-39105863d8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:02.237891 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-356e7913-bc68-4ae4-a411-39105863d8c1 None None] MFA Rules not processed for user `c115cd91f4b144c4a14ac7972a6b9d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:02.239100 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-356e7913-bc68-4ae4-a411-39105863d8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:02.252687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:02.313979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1397/2774] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:21:02.324410 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:02.326510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:02.390067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063edd7-381b-4265-b115-42c3ff3dadb2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:02.393804 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e90180-2afa-4167-b127-7f449f83de5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:02.394458 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e90180-2afa-4167-b127-7f449f83de5a service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:02.394739 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e90180-2afa-4167-b127-7f449f83de5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:02.395010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e90180-2afa-4167-b127-7f449f83de5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:02.401081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:02.406814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e90180-2afa-4167-b127-7f449f83de5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:02.408555 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e90180-2afa-4167-b127-7f449f83de5a service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:02.419361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1378/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:03.083995 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e7913-bc68-4ae4-a411-39105863d8c1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:03.152142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e7913-bc68-4ae4-a411-39105863d8c1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:03.157991 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:03.158884 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:03.159163 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:03.159435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] PATH_INFO: `/v3/users/39bc03745c4142cda024730fd243c552` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:03.160695 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39bc03745c4142cda024730fd243c552)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:03.162736 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:03.210256 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39bc03745c4142cda024730fd243c552'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.210915 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39bc03745c4142cda024730fd243c552'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.211519 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39bc03745c4142cda024730fd243c552'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.223558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39bc03745c4142cda024730fd243c552'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.230374 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39bc03745c4142cda024730fd243c552'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.244658 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1398/2776] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:21:03 2022] DELETE /identity/v3/users/39bc03745c4142cda024730fd243c552 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:03.249605 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:03.250073 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:03.250194 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:03.268383 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:03.269631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:03.333494 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1379/2777] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:03.343407 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:03.345036 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:03.379872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:03.404941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc76292-2e2d-40b2-b246-8527d7ced8d4 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:03.408903 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:03.409589 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:03.409872 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:03.410147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:03.419541 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:03.420962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:03.430831 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1399/2778] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:03.718298 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:03.720032 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:03.733135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87856fec-c8c2-4808-a3c3-88a7b3603d66 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:03.738366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:03.739280 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:03.739562 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:03.739866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] PATH_INFO: `/v3/projects/a8a70384eb524bb08ffc0f6e68861fa9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:03.741107 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a70384eb524bb08ffc0f6e68861fa9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:03.743033 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:03.798968 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a70384eb524bb08ffc0f6e68861fa9', 'request_id': 'req-15505762-5002-4a40-bd38-3525a17d8601'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.808104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a70384eb524bb08ffc0f6e68861fa9', 'request_id': 'req-15505762-5002-4a40-bd38-3525a17d8601'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.815029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a70384eb524bb08ffc0f6e68861fa9', 'request_id': 'req-15505762-5002-4a40-bd38-3525a17d8601'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.827046 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] The token cache is being invalidate because project a8a70384eb524bb08ffc0f6e68861fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:21:03.827291 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.827783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.829504 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1380/2779] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:21:03 2022] DELETE /identity/v3/projects/a8a70384eb524bb08ffc0f6e68861fa9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:03.833613 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:03.844130 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2fa05-a287-4dbb-8f36-a79a83b2896a service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:03.906414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:03.907248 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:03.907525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:03.907803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] PATH_INFO: `/v3/users/c115cd91f4b144c4a14ac7972a6b9d9d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:03.909027 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c115cd91f4b144c4a14ac7972a6b9d9d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:03.910986 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:03.960985 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c115cd91f4b144c4a14ac7972a6b9d9d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.961653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c115cd91f4b144c4a14ac7972a6b9d9d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.962239 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c115cd91f4b144c4a14ac7972a6b9d9d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.974546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c115cd91f4b144c4a14ac7972a6b9d9d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:03.983220 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-10b38a45-197f-4b0d-8bf8-5b37b6b09e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c115cd91f4b144c4a14ac7972a6b9d9d'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:04.006289 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1400/2780] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:21:03 2022] DELETE /identity/v3/users/c115cd91f4b144c4a14ac7972a6b9d9d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:04.300544 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:04.302503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:04.317807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15505762-5002-4a40-bd38-3525a17d8601 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:04.383062 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:04.383919 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.384192 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.384459 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] PATH_INFO: `/v3/projects/459e84940bef4257b1c62611aa3e8cb1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.385646 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=459e84940bef4257b1c62611aa3e8cb1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:04.387578 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:04.408978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:04.440126 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459e84940bef4257b1c62611aa3e8cb1', 'request_id': 'req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:04.449135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459e84940bef4257b1c62611aa3e8cb1', 'request_id': 'req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:04.455909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459e84940bef4257b1c62611aa3e8cb1', 'request_id': 'req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:04.466802 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] The token cache is being invalidate because project 459e84940bef4257b1c62611aa3e8cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:21:04.467081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:04.467745 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d3dcabb4-8f11-4ff8-b662-a9a7070122f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:04.469598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1381/2781] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:21:04 2022] DELETE /identity/v3/projects/459e84940bef4257b1c62611aa3e8cb1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:04.474726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594739f-f3ae-4b9d-855d-6b9f16efcee1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.475224 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594739f-f3ae-4b9d-855d-6b9f16efcee1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.475319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594739f-f3ae-4b9d-855d-6b9f16efcee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.493932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-8594739f-f3ae-4b9d-855d-6b9f16efcee1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:04.495121 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-8594739f-f3ae-4b9d-855d-6b9f16efcee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:04.560048 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:04.577367 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1401/2782] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:04.585560 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a80c8-a562-4dbd-8e79-1d2f7cdd5518 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.585841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a80c8-a562-4dbd-8e79-1d2f7cdd5518 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.586117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a80c8-a562-4dbd-8e79-1d2f7cdd5518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.606001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f98a80c8-a562-4dbd-8e79-1d2f7cdd5518 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:04.607248 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f98a80c8-a562-4dbd-8e79-1d2f7cdd5518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:04.616779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1382/2783] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:04.621159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594739f-f3ae-4b9d-855d-6b9f16efcee1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:04.631638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594739f-f3ae-4b9d-855d-6b9f16efcee1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:04.637633 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:04.638292 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.638575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.638844 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.639579 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:21:04.639888 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:04.641542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:04.654351 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1402/2784] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:21:04.659168 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a80c8-a562-4dbd-8e79-1d2f7cdd5518 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:04.668951 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a80c8-a562-4dbd-8e79-1d2f7cdd5518 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:04.674763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:04.675454 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.675730 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.676001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.676698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:04.678641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:04.708093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1383/2785] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:21:04 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:04.712807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:04.728752 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab844cfe-cbd3-4513-b4f3-8b50946a6e20 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:04.792580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:04.793379 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.793794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.794318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.794586 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:04.796537 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:04.826025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1403/2786] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:21:04 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:04.827289 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:04.842131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3367ee-a3ae-437d-82ce-96c5107437d3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:04.847801 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:04.848495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.848780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.849077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.849846 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:04.851512 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:04.864010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1384/2787] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:04.868615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:04.880615 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc0dbe-7d78-49a2-b433-7c58c646759d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:04.886850 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e19bd1-2637-4b2e-ba02-1de93682a797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:04.887747 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e19bd1-2637-4b2e-ba02-1de93682a797 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.888124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e19bd1-2637-4b2e-ba02-1de93682a797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.888490 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e19bd1-2637-4b2e-ba02-1de93682a797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.889319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e19bd1-2637-4b2e-ba02-1de93682a797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:04.891025 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e19bd1-2637-4b2e-ba02-1de93682a797 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:04.901826 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1404/2788] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:04.906571 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:04.917583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa917-f595-42cf-8261-a35967796d93 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:04.922425 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:04.923323 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.923600 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.923936 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] PATH_INFO: `/v3/projects/c578929ff3234866bc1db6ea2b57403b/users/558578e1272f4f78920446c66caf9ed5/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.940345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c578929ff3234866bc1db6ea2b57403b, user_id=558578e1272f4f78920446c66caf9ed5, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:04.943259 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:04.958867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1385/2789] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:04 2022] PUT /identity/v3/projects/c578929ff3234866bc1db6ea2b57403b/users/558578e1272f4f78920446c66caf9ed5/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:04.970335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453183a-913d-4646-8f44-2d0bf2c8ce65 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:04.970627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453183a-913d-4646-8f44-2d0bf2c8ce65 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:04.970901 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453183a-913d-4646-8f44-2d0bf2c8ce65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:04.999645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-b453183a-913d-4646-8f44-2d0bf2c8ce65 None None] MFA Rules not processed for user `558578e1272f4f78920446c66caf9ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:05.000928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b453183a-913d-4646-8f44-2d0bf2c8ce65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:05.071183 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1405/2790] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:05.084638 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:05.086888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:05.160428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485faf39-ab09-478e-8b86-11bfde58af78 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:05.164503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ae095-202a-44d8-ad3c-578d388e9ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:05.165272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ae095-202a-44d8-ad3c-578d388e9ebd service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:05.165619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ae095-202a-44d8-ad3c-578d388e9ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:05.165961 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ae095-202a-44d8-ad3c-578d388e9ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:05.175111 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ae095-202a-44d8-ad3c-578d388e9ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:05.176550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ae095-202a-44d8-ad3c-578d388e9ebd service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:05.186026 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1386/2791] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:08.255687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453183a-913d-4646-8f44-2d0bf2c8ce65 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:08.311863 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:08.327611 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453183a-913d-4646-8f44-2d0bf2c8ce65 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:08.333318 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:08.334247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:08.334520 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:08.334787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] PATH_INFO: `/v3/users/558578e1272f4f78920446c66caf9ed5` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:08.336147 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=558578e1272f4f78920446c66caf9ed5)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:08.338309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:08.401671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558578e1272f4f78920446c66caf9ed5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:08.404385 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558578e1272f4f78920446c66caf9ed5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:08.404964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558578e1272f4f78920446c66caf9ed5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:08.419277 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558578e1272f4f78920446c66caf9ed5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:08.427425 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558578e1272f4f78920446c66caf9ed5'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:08.442232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1406/2792] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:21:08 2022] DELETE /identity/v3/users/558578e1272f4f78920446c66caf9ed5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:08.446522 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:08.446994 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:08.447104 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:08.464737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:08.466122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:08.508667 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:08.527618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1387/2793] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:21:08.537456 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:08.538964 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:08.609524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51be2f8-9d6f-457c-b5c1-4c0d7e357a09 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:08.614449 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457cde7-2ab5-46f5-bb80-43262fb12173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:08.615328 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457cde7-2ab5-46f5-bb80-43262fb12173 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:08.615646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457cde7-2ab5-46f5-bb80-43262fb12173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:08.616015 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457cde7-2ab5-46f5-bb80-43262fb12173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:08.628912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457cde7-2ab5-46f5-bb80-43262fb12173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:08.630733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457cde7-2ab5-46f5-bb80-43262fb12173 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:08.642443 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1407/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:08.952887 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:08.954934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:08.967135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c887c-100e-47b0-8bc2-a92498050aae None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:08.972117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:08.972987 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:08.973310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:08.973548 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] PATH_INFO: `/v3/projects/c578929ff3234866bc1db6ea2b57403b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:08.974777 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c578929ff3234866bc1db6ea2b57403b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:08.976789 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.038258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c578929ff3234866bc1db6ea2b57403b', 'request_id': 'req-c9a17be2-7a15-4251-8b47-6d3bcecf2571'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:09.046618 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c578929ff3234866bc1db6ea2b57403b', 'request_id': 'req-c9a17be2-7a15-4251-8b47-6d3bcecf2571'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:09.052963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c578929ff3234866bc1db6ea2b57403b', 'request_id': 'req-c9a17be2-7a15-4251-8b47-6d3bcecf2571'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:09.063147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] The token cache is being invalidate because project c578929ff3234866bc1db6ea2b57403b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:21:09.063436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:09.064044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9a17be2-7a15-4251-8b47-6d3bcecf2571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:09.065648 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1388/2795] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:21:08 2022] DELETE /identity/v3/projects/c578929ff3234866bc1db6ea2b57403b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:09.070532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4114a49-1506-4b55-8929-5376ed92e3d7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.070818 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4114a49-1506-4b55-8929-5376ed92e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.071095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4114a49-1506-4b55-8929-5376ed92e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.088714 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d4114a49-1506-4b55-8929-5376ed92e3d7 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:09.089941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d4114a49-1506-4b55-8929-5376ed92e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:09.153581 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1408/2796] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:09.161299 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba622c4f-e9b9-4443-a736-b9fd21fd3659 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.161559 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba622c4f-e9b9-4443-a736-b9fd21fd3659 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.161830 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba622c4f-e9b9-4443-a736-b9fd21fd3659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.183077 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ba622c4f-e9b9-4443-a736-b9fd21fd3659 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:09.184575 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ba622c4f-e9b9-4443-a736-b9fd21fd3659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:09.194315 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1389/2797] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:09.199745 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4114a49-1506-4b55-8929-5376ed92e3d7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:09.210993 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4114a49-1506-4b55-8929-5376ed92e3d7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:09.215923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:09.216597 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.216881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.217196 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.217931 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:21:09.218240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:09.219945 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.227269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:09.237820 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1409/2798] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:21:09.238965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba622c4f-e9b9-4443-a736-b9fd21fd3659 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:09.248273 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba622c4f-e9b9-4443-a736-b9fd21fd3659 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:09.253209 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:09.253869 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.254147 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.254421 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.255131 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:09.257037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.287749 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1390/2799] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:21:09 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:21:09.292086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:09.302925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdff320-3d49-46eb-b657-bfbb280c3684 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:09.368153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:09.368153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.368153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.368153 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.369010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:09.369010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.392719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1410/2800] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:21:09 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:09.394352 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:09.405635 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20a45-03b9-44f1-825e-18a92bd8f38f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:09.411127 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:09.411836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.412114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.412366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.413162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:09.414743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.426093 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1391/2801] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:09.430151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:09.440697 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c0172-1a1c-4028-92e7-f95a07fe4d2d admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:09.446265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4014b85-7fcd-416c-b615-8f216ff2e2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:09.446933 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4014b85-7fcd-416c-b615-8f216ff2e2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.447231 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4014b85-7fcd-416c-b615-8f216ff2e2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.447518 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4014b85-7fcd-416c-b615-8f216ff2e2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.448255 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4014b85-7fcd-416c-b615-8f216ff2e2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:09.450150 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4014b85-7fcd-416c-b615-8f216ff2e2d3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.460627 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1411/2802] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:09.464725 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:09.475639 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429910b-325b-4ba1-b9cd-c37c1279d6b2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:09.480499 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:09.481428 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.481709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.481981 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] PATH_INFO: `/v3/projects/4ba27eb979944a96a89df10c7e0b6dca/users/06e149f2b896405d863c98530333392b/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.498106 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba27eb979944a96a89df10c7e0b6dca, user_id=06e149f2b896405d863c98530333392b, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:09.500743 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.516609 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1392/2803] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:09 2022] PUT /identity/v3/projects/4ba27eb979944a96a89df10c7e0b6dca/users/06e149f2b896405d863c98530333392b/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:09.526754 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.527464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.528042 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.555179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] MFA Rules not processed for user `06e149f2b896405d863c98530333392b`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:09.556307 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:09.635103 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1412/2804] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:09.646087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:09.647874 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:09.706334 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b9dab-bcd1-4042-a62c-46e35a7d46be admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:09.709950 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81c5b1-d686-4185-97d2-33ae72248add service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:09.710598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81c5b1-d686-4185-97d2-33ae72248add service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:09.710879 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81c5b1-d686-4185-97d2-33ae72248add service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:09.711152 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81c5b1-d686-4185-97d2-33ae72248add service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:09.719533 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81c5b1-d686-4185-97d2-33ae72248add service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:09.720927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81c5b1-d686-4185-97d2-33ae72248add service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:09.729900 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1393/2805] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:10.921715 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:10.923221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:11.022667 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:11.052676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc869fd-a521-45fc-a0a9-dcc8e862815b None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:11.058313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b037a8-db60-40a8-be1b-203e4c99d652 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:11.059314 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b037a8-db60-40a8-be1b-203e4c99d652 service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:11.059719 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b037a8-db60-40a8-be1b-203e4c99d652 service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:11.060087 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b037a8-db60-40a8-be1b-203e4c99d652 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:11.062031 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-45b037a8-db60-40a8-be1b-203e4c99d652 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:11.075523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b037a8-db60-40a8-be1b-203e4c99d652 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:11.077154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b037a8-db60-40a8-be1b-203e4c99d652 service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:11.087555 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1413/2806] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:11.172570 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9acc1-0577-4a99-8723-59008ce409a7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:11.173170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9acc1-0577-4a99-8723-59008ce409a7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:11.173170 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9acc1-0577-4a99-8723-59008ce409a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:11.191798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7ac9acc1-0577-4a99-8723-59008ce409a7 None None] MFA Rules not processed for user `06e149f2b896405d863c98530333392b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:11.193002 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7ac9acc1-0577-4a99-8723-59008ce409a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:11.200737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1394/2807] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:17.733895 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac9cd2-fdaf-422d-9507-2ad7ede41346 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:17.734464 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac9cd2-fdaf-422d-9507-2ad7ede41346 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:17.734574 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac9cd2-fdaf-422d-9507-2ad7ede41346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:17.755822 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6fac9cd2-fdaf-422d-9507-2ad7ede41346 None None] MFA Rules not processed for user `1aff23c6c4f6473eba8c6bc659345ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:17.757360 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6fac9cd2-fdaf-422d-9507-2ad7ede41346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:17.824942 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1414/2808] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:21.186436 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:23.605530 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa9aad-e078-4c80-9c91-ed122fcea6a1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:23.606224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa9aad-e078-4c80-9c91-ed122fcea6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:23.606224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa9aad-e078-4c80-9c91-ed122fcea6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:23.625396 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cfaa9aad-e078-4c80-9c91-ed122fcea6a1 None None] MFA Rules not processed for user `7e060034139f420e8412b8a55f1b29a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:23.626999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cfaa9aad-e078-4c80-9c91-ed122fcea6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:23.684629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1395/2809] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:24.585279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:24.586734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:24.587179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:24.607522 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] MFA Rules not processed for user `5279c7bf349746b4a4622f72945662ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:24.608934 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:24.713090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1415/2810] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:33.677738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=25 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:34.701788 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=61 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:35.366546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a28cb5-4623-431f-90b8-082fa857df79 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:35.367029 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a28cb5-4623-431f-90b8-082fa857df79 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:35.367227 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a28cb5-4623-431f-90b8-082fa857df79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:35.396933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-08a28cb5-4623-431f-90b8-082fa857df79 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:35.398225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-08a28cb5-4623-431f-90b8-082fa857df79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:35.466023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1396/2811] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:21:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:35.517838 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:35.519668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:35.593857 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2051dd-d81a-4ac8-92d4-e54d394b953f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:35.599542 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:35.600122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:35.600403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:35.600678 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:35.609275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:35.610873 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:35.620214 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1416/2812] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:21:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:21:36.163687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97422ae-413d-4f0c-a42d-98fff24dda12 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:36.164004 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97422ae-413d-4f0c-a42d-98fff24dda12 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:36.164310 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97422ae-413d-4f0c-a42d-98fff24dda12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:36.182580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c97422ae-413d-4f0c-a42d-98fff24dda12 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:36.183788 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c97422ae-413d-4f0c-a42d-98fff24dda12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:36.192322 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1397/2813] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:36.204500 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:36.215705 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe271-e570-4b05-a4a6-75aa70b66111 service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:36.222151 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:36.222957 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:36.223358 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:36.223636 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:36.232398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:36.234010 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:36.242638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1417/2814] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:21:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:21:36.458211 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf797-5c2c-414b-9a54-7091dd0da67b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:36.458683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf797-5c2c-414b-9a54-7091dd0da67b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:36.458798 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf797-5c2c-414b-9a54-7091dd0da67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:36.476626 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-eb4bf797-5c2c-414b-9a54-7091dd0da67b None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:36.477850 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eb4bf797-5c2c-414b-9a54-7091dd0da67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:36.485368 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1398/2815] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:36.497696 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:36.511509 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26d7fe-73a4-4a50-aa4f-5563329d0955 service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:36.517511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:36.518125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:36.518417 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:36.518681 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:36.528676 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:36.530368 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:36.539083 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1418/2816] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:21:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 07:21:36.888069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f99b3-257e-4b98-a089-08b0a247831a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:36.888069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f99b3-257e-4b98-a089-08b0a247831a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:36.888069 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f99b3-257e-4b98-a089-08b0a247831a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:36.911989 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d06f99b3-257e-4b98-a089-08b0a247831a None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:36.913517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d06f99b3-257e-4b98-a089-08b0a247831a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:36.921365 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1399/2817] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 07:21:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:36.934110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:36.945932 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629b07-41cb-41bf-8ebf-b5947bd736bf service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:36.952660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff215e85-3e14-40f1-bd8e-876bd2fe521e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:36.952660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff215e85-3e14-40f1-bd8e-876bd2fe521e service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:36.952660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff215e85-3e14-40f1-bd8e-876bd2fe521e service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:36.952660 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff215e85-3e14-40f1-bd8e-876bd2fe521e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:36.962063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff215e85-3e14-40f1-bd8e-876bd2fe521e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:36.963675 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff215e85-3e14-40f1-bd8e-876bd2fe521e service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:36.973382 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1419/2818] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:21:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:21:39.112186 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f99b3-257e-4b98-a089-08b0a247831a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:39.187998 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f99b3-257e-4b98-a089-08b0a247831a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:39.192893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:39.193814 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:39.194295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:39.194574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] PATH_INFO: `/v3/users/06e149f2b896405d863c98530333392b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:39.195795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e149f2b896405d863c98530333392b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:39.197871 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:39.249264 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e149f2b896405d863c98530333392b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.250135 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e149f2b896405d863c98530333392b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.263683 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e149f2b896405d863c98530333392b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.264521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e149f2b896405d863c98530333392b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.271121 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:39.272521 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e149f2b896405d863c98530333392b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.292132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1400/2819] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:21:39 2022] DELETE /identity/v3/users/06e149f2b896405d863c98530333392b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:39.296973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:39.297525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:39.297733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:39.316305 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:39.318043 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:39.378030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1420/2820] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:21:39.387791 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:39.389217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:39.490751 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525fc7a-80ed-4f1b-a847-71233c8cd4b4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:39.494797 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781786b-c359-4fb3-85e2-fb695e699b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:39.495643 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781786b-c359-4fb3-85e2-fb695e699b07 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:39.496114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781786b-c359-4fb3-85e2-fb695e699b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:39.496357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781786b-c359-4fb3-85e2-fb695e699b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:39.512867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781786b-c359-4fb3-85e2-fb695e699b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:39.514331 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781786b-c359-4fb3-85e2-fb695e699b07 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:39.525316 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1401/2821] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:39.720034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:39.722111 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:39.742782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc5fd5-5508-438a-ba11-f0c7e9437a11 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:39.749575 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:39.752309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:39.752962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:39.753325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] PATH_INFO: `/v3/projects/4ba27eb979944a96a89df10c7e0b6dca` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:39.755142 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba27eb979944a96a89df10c7e0b6dca)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:39.757283 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:39.815730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:39.819548 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba27eb979944a96a89df10c7e0b6dca', 'request_id': 'req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.827060 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba27eb979944a96a89df10c7e0b6dca', 'request_id': 'req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.835110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba27eb979944a96a89df10c7e0b6dca', 'request_id': 'req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.852440 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] The token cache is being invalidate because project 4ba27eb979944a96a89df10c7e0b6dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:21:39.852819 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.853614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-27a6cf42-0fa2-4075-9258-731e9a8c5cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:21:39.856213 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1421/2822] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:21:39 2022] DELETE /identity/v3/projects/4ba27eb979944a96a89df10c7e0b6dca => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:39.861360 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac810c-3297-471a-9290-789f0759ae00 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:39.861650 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac810c-3297-471a-9290-789f0759ae00 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:39.862123 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac810c-3297-471a-9290-789f0759ae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:39.890541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1eac810c-3297-471a-9290-789f0759ae00 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:39.892132 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1eac810c-3297-471a-9290-789f0759ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:39.969685 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1402/2823] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:21:39.977768 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df3b5-373d-4f6b-9b21-b97060f56e11 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:39.978053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df3b5-373d-4f6b-9b21-b97060f56e11 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:39.978323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df3b5-373d-4f6b-9b21-b97060f56e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.001469 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-d45df3b5-373d-4f6b-9b21-b97060f56e11 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:40.002789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d45df3b5-373d-4f6b-9b21-b97060f56e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:40.011956 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1422/2824] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:40.019737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac810c-3297-471a-9290-789f0759ae00 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:40.028177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac810c-3297-471a-9290-789f0759ae00 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:40.032941 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:40.033552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.033857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.034100 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.034836 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:21:40.035141 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:40.036993 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:40.054388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1403/2825] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 07:21:40.060112 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df3b5-373d-4f6b-9b21-b97060f56e11 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:40.074144 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df3b5-373d-4f6b-9b21-b97060f56e11 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:40.077789 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:40.078432 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.078905 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.079614 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.080858 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:40.083263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:40.115741 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1423/2826] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:21:40 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:40.120037 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:40.134738 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d25970-3188-402c-a941-9d3115ea4d44 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:40.201071 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:40.202076 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.202361 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.202965 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.203728 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:40.205808 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:40.214177 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:40.237112 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1404/2827] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:21:40 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 07:21:40.240337 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:40.251743 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bf543-f563-4960-abb1-2db8b03d12fd admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:40.257433 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:40.258120 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.258397 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.258674 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.259409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:40.261026 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:40.273505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1424/2828] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:40.278277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:40.290451 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1aecc-6fb3-4ade-9737-1a2da478b3e9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:40.298366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accabe9-59aa-4118-b3cf-9a94e6dfe937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:40.299128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accabe9-59aa-4118-b3cf-9a94e6dfe937 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.299580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accabe9-59aa-4118-b3cf-9a94e6dfe937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.300010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accabe9-59aa-4118-b3cf-9a94e6dfe937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.301197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accabe9-59aa-4118-b3cf-9a94e6dfe937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:40.303146 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accabe9-59aa-4118-b3cf-9a94e6dfe937 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:40.317899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1405/2829] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:40.322309 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:40.344191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5780bc0-82d8-488d-9025-5d44c414a284 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:40.351107 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:40.352125 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] REQUEST_METHOD: `PUT` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.352643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.353110 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] PATH_INFO: `/v3/projects/34750fb4357248b7a99cc1f0baaad4ae/users/e824ff37e1724f549f1e113013c554fe/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.371233 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34750fb4357248b7a99cc1f0baaad4ae, user_id=e824ff37e1724f549f1e113013c554fe, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:40.375406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:40.393228 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1425/2830] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:40 2022] PUT /identity/v3/projects/34750fb4357248b7a99cc1f0baaad4ae/users/e824ff37e1724f549f1e113013c554fe/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:21:40.399460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71fc6c-4eeb-4368-b3fb-23b911b07eb1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.399812 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71fc6c-4eeb-4368-b3fb-23b911b07eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.400151 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71fc6c-4eeb-4368-b3fb-23b911b07eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.421237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ae71fc6c-4eeb-4368-b3fb-23b911b07eb1 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:40.422696 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ae71fc6c-4eeb-4368-b3fb-23b911b07eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:40.487117 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1406/2831] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:21:40.496979 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:40.498946 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:40.559003 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8faf9-a43c-469d-ade5-2a8cc09a218a admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:40.563240 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:40.563882 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:40.564162 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:40.564435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:40.575501 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:40.576912 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:40.586913 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1426/2832] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:44.530811 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9cce4-ed8b-43c9-8b35-e6cdce61b134 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:44.531351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9cce4-ed8b-43c9-8b35-e6cdce61b134 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:44.531351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9cce4-ed8b-43c9-8b35-e6cdce61b134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:44.554210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9ab9cce4-ed8b-43c9-8b35-e6cdce61b134 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:44.555763 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9ab9cce4-ed8b-43c9-8b35-e6cdce61b134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:44.625906 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1407/2833] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:21:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:44.640244 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:44.641507 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:44.751410 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad3da4-cec2-424c-af17-04cd3030846a service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:44.759649 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:44.760428 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:44.760779 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:44.761313 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:44.773409 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:44.775154 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:44.798906 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1427/2834] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:21:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:21:44.984439 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93c31-2fcf-4b38-87db-9ba8dfd253b8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:44.984737 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93c31-2fcf-4b38-87db-9ba8dfd253b8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:44.985017 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93c31-2fcf-4b38-87db-9ba8dfd253b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:45.059940 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6bb93c31-2fcf-4b38-87db-9ba8dfd253b8 None None] MFA Rules not processed for user `e824ff37e1724f549f1e113013c554fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:45.063195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6bb93c31-2fcf-4b38-87db-9ba8dfd253b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:45.167828 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1408/2835] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:45.173218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:45.179918 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:45.201445 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce62d9-600e-4525-9395-fb9f490dda88 service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:45.213896 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:45.214707 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:45.215299 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:45.215447 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:45.216269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:45.220491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:45.273643 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1428/2836] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:21:45 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:45.275028 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb93c31-2fcf-4b38-87db-9ba8dfd253b8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:45.294496 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb93c31-2fcf-4b38-87db-9ba8dfd253b8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:45.309236 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:45.370519 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:45.371207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:45.371490 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:45.371764 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:45.372488 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:45.374572 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:45.409517 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1409/2837] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:21:45 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:21:45.411558 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:45.426457 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740abb-677f-4ff2-be9f-598cc5fe6084 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:45.432369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:45.433078 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:45.433512 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:45.433828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:45.434655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:45.436282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:45.469904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1429/2838] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:45.474641 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:45.488983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f3d7-bb7d-4372-a5c3-7a57b6c77d37 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:45.496218 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:45.497208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:45.497489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:45.497775 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] PATH_INFO: `/v3/projects/c2125fad30014073968b70869e7372f3/users/b6af6ba9a232484280d672963c664ee0/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:45.520620 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2125fad30014073968b70869e7372f3, user_id=b6af6ba9a232484280d672963c664ee0, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:45.523314 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:45.549715 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1410/2839] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:45 2022] PUT /identity/v3/projects/c2125fad30014073968b70869e7372f3/users/b6af6ba9a232484280d672963c664ee0/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:45.553803 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:45.569170 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae8abe-be90-4c1f-89e9-1f3a2a73da42 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:45.670004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:45.671286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:45.671286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:45.671532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:45.672247 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:45.674471 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:45.692866 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:45.700138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1430/2840] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:21:45.716631 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f1404-07a2-4383-aad4-34e5998cf37f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:45.724254 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02545570-e293-4bcd-8d19-2817360c2ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:45.725297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545570-e293-4bcd-8d19-2817360c2ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:45.727913 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545570-e293-4bcd-8d19-2817360c2ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:45.728224 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545570-e293-4bcd-8d19-2817360c2ddd admin admin] PATH_INFO: `/v3/projects/c2125fad30014073968b70869e7372f3/users/b6af6ba9a232484280d672963c664ee0/roles/aaca343d2e4a43c88674834f08499f15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:45.732128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02545570-e293-4bcd-8d19-2817360c2ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2125fad30014073968b70869e7372f3, user_id=b6af6ba9a232484280d672963c664ee0, role_id=aaca343d2e4a43c88674834f08499f15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:45.743114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02545570-e293-4bcd-8d19-2817360c2ddd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:45.762501 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1411/2841] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:21:45 2022] PUT /identity/v3/projects/c2125fad30014073968b70869e7372f3/users/b6af6ba9a232484280d672963c664ee0/roles/aaca343d2e4a43c88674834f08499f15 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:21:46.757016 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:46.877009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fe7a7-887e-4f2a-9926-c63622907072 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:46.996124 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:46.997810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:46.998334 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:46.998876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:47.001505 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:47.088737 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:47.173502 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:47.191199 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:47.248171 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1431/2842] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:54.264006 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc041c-1e93-4662-a22a-5c3c14e790ba None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:54.264547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc041c-1e93-4662-a22a-5c3c14e790ba None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:54.264547 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc041c-1e93-4662-a22a-5c3c14e790ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:54.305596 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bebc041c-1e93-4662-a22a-5c3c14e790ba None None] MFA Rules not processed for user `b6af6ba9a232484280d672963c664ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:54.306986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bebc041c-1e93-4662-a22a-5c3c14e790ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:54.434176 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1412/2843] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:21:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:54.452653 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:54.454286 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:54.539904 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b7a89-26a0-4c11-b3c7-052d2a754e6e service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:54.550460 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366a7e7596d14b32a6ce62790a2e3dd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:54.552180 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:54.552680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:54.553263 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:54.632656 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:54.634253 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:54.647990 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1432/2844] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:21:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:21:54.781305 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd69cd-ce2e-49f2-b3bf-db5d6035175c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:54.781807 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd69cd-ce2e-49f2-b3bf-db5d6035175c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:54.782309 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd69cd-ce2e-49f2-b3bf-db5d6035175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:54.810476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-21cd69cd-ce2e-49f2-b3bf-db5d6035175c None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:21:54.812067 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-21cd69cd-ce2e-49f2-b3bf-db5d6035175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:21:54.867955 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:21:54.881629 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1413/2845] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:21:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:21:54.905845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:21:55.025040 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe1996-558e-4633-b8ac-bda9cfcc3bb3 service glance] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:21:55.039580 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848db3d-5ca3-47f7-a202-9d095ec8849e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:21:55.040546 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848db3d-5ca3-47f7-a202-9d095ec8849e service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:21:55.041221 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848db3d-5ca3-47f7-a202-9d095ec8849e service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:21:55.041750 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848db3d-5ca3-47f7-a202-9d095ec8849e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:21:55.052491 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848db3d-5ca3-47f7-a202-9d095ec8849e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:21:55.054869 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848db3d-5ca3-47f7-a202-9d095ec8849e service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:21:55.079138 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1433/2846] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:21:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:22:04.878818 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:05.060398 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=93 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:06.271277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744a34-e1dd-473c-913c-ec4bb8abf37c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:06.271852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744a34-e1dd-473c-913c-ec4bb8abf37c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:06.271852 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744a34-e1dd-473c-913c-ec4bb8abf37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:06.290020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-60744a34-e1dd-473c-913c-ec4bb8abf37c None None] MFA Rules not processed for user `e824ff37e1724f549f1e113013c554fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:22:06.291217 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-60744a34-e1dd-473c-913c-ec4bb8abf37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:06.298550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1414/2847] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:22:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:22:06.979642 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:06.980204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:06.980204 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:07.010280 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:22:07.011655 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:07.019923 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1434/2848] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 07:22:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:22:07.036388 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744a34-e1dd-473c-913c-ec4bb8abf37c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:07.051633 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744a34-e1dd-473c-913c-ec4bb8abf37c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:07.056619 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8944cdb-9bbb-434d-a97a-9408fc114753 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:07.057328 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8944cdb-9bbb-434d-a97a-9408fc114753 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:07.057632 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8944cdb-9bbb-434d-a97a-9408fc114753 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:07.058122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8944cdb-9bbb-434d-a97a-9408fc114753 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:07.067742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8944cdb-9bbb-434d-a97a-9408fc114753 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:07.069453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8944cdb-9bbb-434d-a97a-9408fc114753 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:07.089628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1415/2849] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:22:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:22:11.184009 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:11.185917 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:11.280007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8914b0-55d1-4063-8b44-b1750a697133 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:11.285761 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:11.286871 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:11.287406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:11.287807 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] PATH_INFO: `/v3/users/1aff23c6c4f6473eba8c6bc659345ba9` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:11.289326 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aff23c6c4f6473eba8c6bc659345ba9)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:11.291709 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:11.376875 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aff23c6c4f6473eba8c6bc659345ba9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.377738 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aff23c6c4f6473eba8c6bc659345ba9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.380711 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aff23c6c4f6473eba8c6bc659345ba9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.401939 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aff23c6c4f6473eba8c6bc659345ba9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.409488 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aff23c6c4f6473eba8c6bc659345ba9'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.441119 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1435/2850] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:22:11 2022] DELETE /identity/v3/users/1aff23c6c4f6473eba8c6bc659345ba9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:22:11.448557 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:11.449592 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:11.450108 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:11.474081 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:22:11.476717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:11.560943 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1416/2851] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 07:22:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:22:11.576033 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:11.578369 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:11.699149 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:11.700932 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:11.753806 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bdb3f-a834-41d9-98d3-f791eef22ea3 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:11.760288 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fd819-26b4-429e-9bed-70716042a328 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:11.761166 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fd819-26b4-429e-9bed-70716042a328 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:11.761644 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fd819-26b4-429e-9bed-70716042a328 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:11.762349 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fd819-26b4-429e-9bed-70716042a328 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:11.773403 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fd819-26b4-429e-9bed-70716042a328 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:11.776129 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fd819-26b4-429e-9bed-70716042a328 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:11.789293 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1436/2852] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:22:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:22:11.812270 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11bf3-f4cc-4fe8-9875-9e586ca09564 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:11.817345 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:11.818327 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:11.818700 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:11.818977 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] PATH_INFO: `/v3/projects/3f78fe1ecaca42d2ae09999dcb3c8d7c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:11.820495 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f78fe1ecaca42d2ae09999dcb3c8d7c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:11.822698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:11.901442 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f78fe1ecaca42d2ae09999dcb3c8d7c', 'request_id': 'req-3c1ef997-1ddd-4df9-91e6-8298de617b40'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.913720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f78fe1ecaca42d2ae09999dcb3c8d7c', 'request_id': 'req-3c1ef997-1ddd-4df9-91e6-8298de617b40'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.929313 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f78fe1ecaca42d2ae09999dcb3c8d7c', 'request_id': 'req-3c1ef997-1ddd-4df9-91e6-8298de617b40'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.950901 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] The token cache is being invalidate because project 3f78fe1ecaca42d2ae09999dcb3c8d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:22:11.951446 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.952237 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3c1ef997-1ddd-4df9-91e6-8298de617b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:11.962561 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1417/2853] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:22:11 2022] DELETE /identity/v3/projects/3f78fe1ecaca42d2ae09999dcb3c8d7c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:22:11.966485 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573009c-db0a-48c1-a1c2-6789b882341f None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:11.966841 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573009c-db0a-48c1-a1c2-6789b882341f None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:11.968218 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573009c-db0a-48c1-a1c2-6789b882341f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:11.984583 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:12.022132 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-6573009c-db0a-48c1-a1c2-6789b882341f None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:22:12.024135 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6573009c-db0a-48c1-a1c2-6789b882341f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:12.162557 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1437/2854] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:22:12.170867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2d45-0654-4315-b8da-7dbcaa397833 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.171652 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2d45-0654-4315-b8da-7dbcaa397833 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.172568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2d45-0654-4315-b8da-7dbcaa397833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.212434 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-339f2d45-0654-4315-b8da-7dbcaa397833 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:22:12.214621 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-339f2d45-0654-4315-b8da-7dbcaa397833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:12.225498 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1418/2855] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:22:12.229680 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573009c-db0a-48c1-a1c2-6789b882341f None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:12.242191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573009c-db0a-48c1-a1c2-6789b882341f None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:12.248475 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:12.249195 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.249476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.249746 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] PATH_INFO: `/v3/domains` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.250486 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80473) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 07:22:12.250794 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:12.252989 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:12.267834 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1438/2856] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 07:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 07:22:12.272986 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2d45-0654-4315-b8da-7dbcaa397833 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:12.288545 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2d45-0654-4315-b8da-7dbcaa397833 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:12.293996 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:12.295552 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.295978 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.296320 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.297208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:12.299189 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:12.304462 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:12.351119 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1419/2857] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 07:22:12 2022] POST /identity/v3/projects => generated 369 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:22:12.355279 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:12.366733 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f39dc-3229-49bc-b03a-a9e358e6aaff admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:12.443438 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:12.445143 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.445489 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.445777 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] PATH_INFO: `/v3/users` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.447435 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:12.449621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:12.484302 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1439/2858] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 07:22:12 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 07:22:12.486013 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:12.499532 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc1e34-b531-4de5-adac-316bf7f4d3c4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:12.509381 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:12.512238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.512238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.512238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.512238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:12.513465 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:12.530122 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:12.530847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1420/2859] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:22:12.541405 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b558c22-407a-4773-886b-18bdadef7190 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:12.547431 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbb82c-2d36-48cb-814a-83dc59474a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:12.547723 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbb82c-2d36-48cb-814a-83dc59474a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.548206 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbb82c-2d36-48cb-814a-83dc59474a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.548576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbb82c-2d36-48cb-814a-83dc59474a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.549840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbb82c-2d36-48cb-814a-83dc59474a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:12.552104 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbb82c-2d36-48cb-814a-83dc59474a18 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:12.566159 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1440/2860] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 07:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 07:22:12.570134 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:12.585172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6588b-47f6-467d-9d9d-6e6a54f42046 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:12.605255 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:12.606574 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.607048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.607541 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] PATH_INFO: `/v3/projects/dad8b14c26044ec9953bc00a5452242d/users/f38ae31fcde4408abd9d3fd039ec2e4f/roles/9b4fe27dba204b05b692207de4fcedd4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.638853 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad8b14c26044ec9953bc00a5452242d, user_id=f38ae31fcde4408abd9d3fd039ec2e4f, role_id=9b4fe27dba204b05b692207de4fcedd4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:12.642172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:12.659196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1421/2861] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 07:22:12 2022] PUT /identity/v3/projects/dad8b14c26044ec9953bc00a5452242d/users/f38ae31fcde4408abd9d3fd039ec2e4f/roles/9b4fe27dba204b05b692207de4fcedd4 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:22:12.669399 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.669687 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.669958 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.697881 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] MFA Rules not processed for user `f38ae31fcde4408abd9d3fd039ec2e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:22:12.699095 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:12.796860 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:12.807859 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1441/2862] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:22:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:22:12.819044 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:12.820824 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:12.926982 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b80289-6519-47db-b1ed-4c8ff335c5e6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:12.931477 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '366a7e7596d14b32a6ce62790a2e3dd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:12.932363 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:12.932780 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:12.933190 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:12.942838 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:12.944546 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:12.957082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1422/2863] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:22:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:22:22.814179 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:22.951122 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:30.930728 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:30.932389 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:31.003062 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a837ca-1f86-4b0d-8c15-e237c8abadb1 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:31.008281 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:31.009227 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:31.009515 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:31.009809 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] PATH_INFO: `/v3/users/7e060034139f420e8412b8a55f1b29a2` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:31.011186 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e060034139f420e8412b8a55f1b29a2)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:31.013198 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:31.065076 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e060034139f420e8412b8a55f1b29a2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.065720 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e060034139f420e8412b8a55f1b29a2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.066335 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e060034139f420e8412b8a55f1b29a2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.079098 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e060034139f420e8412b8a55f1b29a2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.086539 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e060034139f420e8412b8a55f1b29a2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.100689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1442/2864] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:22:30 2022] DELETE /identity/v3/users/7e060034139f420e8412b8a55f1b29a2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:22:31.287000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:31.298795 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f8f38-dbcd-48e8-baa0-e241edcbeb96 service glance] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:31.358001 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:31.358945 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:31.359214 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:31.359487 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] PATH_INFO: `/v3/projects/f15d014b88a444c28ae32f230d144e8c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:31.360723 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15d014b88a444c28ae32f230d144e8c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:31.362722 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:31.412293 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15d014b88a444c28ae32f230d144e8c', 'request_id': 'req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.421207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15d014b88a444c28ae32f230d144e8c', 'request_id': 'req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.428359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15d014b88a444c28ae32f230d144e8c', 'request_id': 'req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.440166 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] The token cache is being invalidate because project f15d014b88a444c28ae32f230d144e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:22:31.440453 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.441357 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:31.443354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1423/2865] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:22:31 2022] DELETE /identity/v3/projects/f15d014b88a444c28ae32f230d144e8c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 07:22:37.414523 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:37.415962 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:37.473907 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eefbad-ece7-40cd-a68d-8f6fbb31f341 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:37.507617 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:37.526014 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1c121-ea58-466c-9ebd-0f8ffae878f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:37.526666 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c121-ea58-466c-9ebd-0f8ffae878f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:37.526943 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c121-ea58-466c-9ebd-0f8ffae878f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:37.527229 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c121-ea58-466c-9ebd-0f8ffae878f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:37.529030 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-03e1c121-ea58-466c-9ebd-0f8ffae878f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:37.595191 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c121-ea58-466c-9ebd-0f8ffae878f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:37.596634 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c121-ea58-466c-9ebd-0f8ffae878f8 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:37.606692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1443/2866] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:22:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:22:41.436847 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:47.568876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:47.570607 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:47.602086 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=34 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:47.637610 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77e8cf-0075-4a4c-9cdb-c8e00ee10afb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:47.643095 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:47.643959 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:47.644226 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:47.644510 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] PATH_INFO: `/v3/users/f38ae31fcde4408abd9d3fd039ec2e4f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:47.645827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38ae31fcde4408abd9d3fd039ec2e4f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:47.647720 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:47.696640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38ae31fcde4408abd9d3fd039ec2e4f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:47.697343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38ae31fcde4408abd9d3fd039ec2e4f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:47.711099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38ae31fcde4408abd9d3fd039ec2e4f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:47.711891 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38ae31fcde4408abd9d3fd039ec2e4f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:47.719210 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38ae31fcde4408abd9d3fd039ec2e4f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:47.733623 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1424/2867] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:22:47 2022] DELETE /identity/v3/users/f38ae31fcde4408abd9d3fd039ec2e4f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:22:47.738423 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:47.738843 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:47.739203 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:47.757079 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] MFA Rules not processed for user `3921c7a7e4fd41ef9eea262b7a97405a`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:22:47.758303 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:47.819813 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1444/2868] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 07:22:47.832087 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:47.833698 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:47.893709 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0281b13-5ded-42c0-b585-d301387df4ce admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:47.898398 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:47.899042 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:47.899326 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:47.899598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:47.909197 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:47.910693 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:47.920622 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1425/2869] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:22:48.195511 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:22:48.197063 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:22:48.210354 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c244e40-2dbc-44d1-8aa1-0f118470f4b7 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:22:48.216257 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:22:48.217412 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:22:48.217671 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:22:48.218056 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] PATH_INFO: `/v3/projects/dad8b14c26044ec9953bc00a5452242d` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:22:48.219341 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad8b14c26044ec9953bc00a5452242d)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:22:48.221453 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:22:48.270355 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad8b14c26044ec9953bc00a5452242d', 'request_id': 'req-731241d3-6d80-45ac-84a8-eb944be10ca2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:48.276951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad8b14c26044ec9953bc00a5452242d', 'request_id': 'req-731241d3-6d80-45ac-84a8-eb944be10ca2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:48.282842 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad8b14c26044ec9953bc00a5452242d', 'request_id': 'req-731241d3-6d80-45ac-84a8-eb944be10ca2'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:48.297474 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] The token cache is being invalidate because project dad8b14c26044ec9953bc00a5452242d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80473) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:22:48.297780 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:48.298414 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:22:48.300008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1445/2870] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:22:48 2022] DELETE /identity/v3/projects/dad8b14c26044ec9953bc00a5452242d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:22:57.916417 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:22:58.294576 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:23:13.264701 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:23:13.276268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1245e0-3554-4c8c-9502-e5800daf823f service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:23:13.331670 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:23:13.332680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:23:13.332680 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:23:13.332867 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:23:13.334754 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:23:13.420234 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:23:13.421717 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:23:13.443199 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1426/2871] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:23:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:23:14.641560 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:23:14.643689 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:23:14.662702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731241d3-6d80-45ac-84a8-eb944be10ca2 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:23:14.666795 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:23:14.667444 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:23:14.667726 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:23:14.668004 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:23:14.669982 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:23:14.685948 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:23:14.687406 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:23:14.699055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1446/2872] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:23:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:23:23.440762 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=63 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:23:24.694007 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=6 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:24:05.052888 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:05.054471 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:05.066083 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a41c3-8b36-4af6-a8f0-f8c266f0e2c9 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:05.073082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06b5af-bf6b-4338-bd38-613980a27ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:05.073748 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06b5af-bf6b-4338-bd38-613980a27ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:05.074023 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06b5af-bf6b-4338-bd38-613980a27ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:05.074295 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06b5af-bf6b-4338-bd38-613980a27ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:05.076473 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2b06b5af-bf6b-4338-bd38-613980a27ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:05.087433 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06b5af-bf6b-4338-bd38-613980a27ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:05.088876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06b5af-bf6b-4338-bd38-613980a27ea4 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:05.098460 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1427/2873] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:24:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:24:06.077077 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:06.078708 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:06.094055 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89420694-5871-40c7-911c-c28497e01d29 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:06.100832 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:06.101577 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:06.101973 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:06.102260 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:06.104115 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:06.118477 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:06.120164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:06.134756 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1447/2874] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:24:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:24:07.924253 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a063-19f5-41bd-8bfb-68f4460ae9b8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:07.924779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a063-19f5-41bd-8bfb-68f4460ae9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:07.924779 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a063-19f5-41bd-8bfb-68f4460ae9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:07.944809 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9737a063-19f5-41bd-8bfb-68f4460ae9b8 None None] MFA Rules not processed for user `5279c7bf349746b4a4622f72945662ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:24:07.946049 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9737a063-19f5-41bd-8bfb-68f4460ae9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:08.007419 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1428/2875] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:24:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:24:08.843877 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:08.845339 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:08.922900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d10f49-2454-4b45-8f6b-d5c80906ec0b service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:08.929961 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:08.930609 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:08.930889 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:08.931164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:08.933081 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:08.944810 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:08.946411 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:08.956566 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1448/2876] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:24:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:24:17.999109 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=31 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:24:18.952237 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=38 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:24:22.530114 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737a063-19f5-41bd-8bfb-68f4460ae9b8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:22.548579 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737a063-19f5-41bd-8bfb-68f4460ae9b8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:22.561927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e8d42-0ca6-4a63-844c-1dd9bf2484ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:22.562822 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e8d42-0ca6-4a63-844c-1dd9bf2484ce service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:22.563082 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e8d42-0ca6-4a63-844c-1dd9bf2484ce service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:22.563355 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e8d42-0ca6-4a63-844c-1dd9bf2484ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:22.565335 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bf0e8d42-0ca6-4a63-844c-1dd9bf2484ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:22.579012 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e8d42-0ca6-4a63-844c-1dd9bf2484ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:22.580598 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e8d42-0ca6-4a63-844c-1dd9bf2484ce service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:22.592167 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1429/2877] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:24:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:24:23.844492 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:23.846232 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:23.862647 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be05f34-adc2-4d1c-ab5f-d5ad41d4de78 service placement] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:23.868951 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95c860dcbf3b40bbb7427a417357c68a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:23.869645 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:23.869996 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:23.870323 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:23.872134 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:23.886599 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:23.888184 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:23.900053 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1449/2878] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:24:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:24:26.010128 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d9eba-b9ea-45ca-8d02-33ffa7daa8a1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:26.010665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d9eba-b9ea-45ca-8d02-33ffa7daa8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:26.010665 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d9eba-b9ea-45ca-8d02-33ffa7daa8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:26.063703 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-db6d9eba-b9ea-45ca-8d02-33ffa7daa8a1 None None] MFA Rules not processed for user `d901a025bdee47e4800f8a250bef1245`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:24:26.064923 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db6d9eba-b9ea-45ca-8d02-33ffa7daa8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:26.129827 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1430/2879] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 07:24:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:24:26.172352 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:26.174450 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:26.268928 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd0bee-3daf-4a98-92a5-305a0ff27259 service nova] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:26.273949 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de86f0a9c2d14cc0b26b19ed719529ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:26.274606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:26.274900 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:26.275269 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:26.283845 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:26.285494 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:26.294034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1450/2880] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 07:24:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 07:24:30.292898 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d9eba-b9ea-45ca-8d02-33ffa7daa8a1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:30.304963 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d9eba-b9ea-45ca-8d02-33ffa7daa8a1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:30.309886 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:30.310539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:30.310933 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:30.311208 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:30.312967 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:30.324705 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:30.326568 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:30.353124 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1431/2881] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:24:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:24:33.600836 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:33.603023 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:33.679898 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0d1b-4292-49ae-b4e3-7d5b23d37cd3 service swift] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:33.684965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:33.685876 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:33.686160 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:33.686441 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] PATH_INFO: `/v3/users/e824ff37e1724f549f1e113013c554fe` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:33.687646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e824ff37e1724f549f1e113013c554fe)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:33.689621 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:33.739028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e824ff37e1724f549f1e113013c554fe'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:33.739668 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e824ff37e1724f549f1e113013c554fe'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:33.740261 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e824ff37e1724f549f1e113013c554fe'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:33.748346 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:24:33.753342 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e824ff37e1724f549f1e113013c554fe'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:33.760424 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e824ff37e1724f549f1e113013c554fe'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:33.780606 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1451/2882] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:24:33 2022] DELETE /identity/v3/users/e824ff37e1724f549f1e113013c554fe => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:24:33.962047 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:33.975742 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809741d1-0dd0-4e29-8ab1-4e9858e13905 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:34.039351 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:34.040193 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:34.040476 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:34.040783 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] PATH_INFO: `/v3/projects/34750fb4357248b7a99cc1f0baaad4ae` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:34.041999 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34750fb4357248b7a99cc1f0baaad4ae)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:34.044000 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:34.093311 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:24:34.095054 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34750fb4357248b7a99cc1f0baaad4ae', 'request_id': 'req-b37d6398-2a75-42c4-9f46-65a8e1bf133c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.104213 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34750fb4357248b7a99cc1f0baaad4ae', 'request_id': 'req-b37d6398-2a75-42c4-9f46-65a8e1bf133c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.110731 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34750fb4357248b7a99cc1f0baaad4ae', 'request_id': 'req-b37d6398-2a75-42c4-9f46-65a8e1bf133c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.121724 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] The token cache is being invalidate because project 34750fb4357248b7a99cc1f0baaad4ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:24:34.122009 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.122983 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.124543 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1432/2883] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:24:33 2022] DELETE /identity/v3/projects/34750fb4357248b7a99cc1f0baaad4ae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:24:34.128712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:34.130287 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:34.144343 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3620500-f65b-4768-986c-4a2ad7b16ae8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:34.210631 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:34.211532 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:34.211840 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:34.212117 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] PATH_INFO: `/v3/users/b6af6ba9a232484280d672963c664ee0` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:34.213415 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6af6ba9a232484280d672963c664ee0)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:34.215366 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:34.270090 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6af6ba9a232484280d672963c664ee0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.270828 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6af6ba9a232484280d672963c664ee0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.271535 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6af6ba9a232484280d672963c664ee0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.284067 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6af6ba9a232484280d672963c664ee0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.291158 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6af6ba9a232484280d672963c664ee0'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.305944 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1452/2884] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:24:34 2022] DELETE /identity/v3/users/b6af6ba9a232484280d672963c664ee0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:24:34.483672 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:24:34.485191 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:24:34.500687 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6398-2a75-42c4-9f46-65a8e1bf133c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:24:34.561616 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:24:34.562376 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:24:34.562654 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:24:34.562924 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] PATH_INFO: `/v3/projects/c2125fad30014073968b70869e7372f3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:24:34.564099 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2125fad30014073968b70869e7372f3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:24:34.566048 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:24:34.612404 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2125fad30014073968b70869e7372f3', 'request_id': 'req-83408901-54b6-4fc9-8e2e-7cc35be62044'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.622343 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2125fad30014073968b70869e7372f3', 'request_id': 'req-83408901-54b6-4fc9-8e2e-7cc35be62044'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.628851 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2125fad30014073968b70869e7372f3', 'request_id': 'req-83408901-54b6-4fc9-8e2e-7cc35be62044'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.639909 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] The token cache is being invalidate because project c2125fad30014073968b70869e7372f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:24:34.640196 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.640858 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:24:34.642484 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1433/2885] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:24:34 2022] DELETE /identity/v3/projects/c2125fad30014073968b70869e7372f3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:24:44.302637 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:24:44.637140 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:26:05.667965 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:26:05.670481 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:26:05.759959 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450c057-fea1-471e-a14f-b93ee0a3e0d8 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:26:05.809782 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46ce9e-7294-42ad-afb4-51349504ab5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4292042e79c45438b5964710359cd5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:26:05.810463 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ce9e-7294-42ad-afb4-51349504ab5d service cinder] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:26:05.810740 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ce9e-7294-42ad-afb4-51349504ab5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:26:05.811022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ce9e-7294-42ad-afb4-51349504ab5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:26:05.813099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cd46ce9e-7294-42ad-afb4-51349504ab5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:26:05.880210 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46ce9e-7294-42ad-afb4-51349504ab5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:26:05.881684 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46ce9e-7294-42ad-afb4-51349504ab5d service cinder] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:26:05.890878 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1453/2886] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:26:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:26:13.228489 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:26:13.230172 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:26:13.303841 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83408901-54b6-4fc9-8e2e-7cc35be62044 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:26:13.309953 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:26:13.310628 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:26:13.310893 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:26:13.311161 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:26:13.313268 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:26:13.325074 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:26:13.326735 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:26:13.336580 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1434/2887] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:26:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:26:13.846483 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f01d9-18cc-454c-88e8-585272493f44 None None] REQUEST_METHOD: `POST` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:26:13.847000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f01d9-18cc-454c-88e8-585272493f44 None None] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:26:13.847319 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f01d9-18cc-454c-88e8-585272493f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:26:13.865975 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.auth.core [None req-cd7f01d9-18cc-454c-88e8-585272493f44 None None] MFA Rules not processed for user `5279c7bf349746b4a4622f72945662ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80473) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 07:26:13.867275 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-cd7f01d9-18cc-454c-88e8-585272493f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:26:13.875108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1454/2888] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 07:26:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 07:26:14.200179 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:26:14.201816 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:26:14.263391 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3c3e3-2a58-44b1-a73b-aa724da7c374 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:26:14.267939 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:26:14.268617 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:26:14.268899 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:26:14.269204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:26:14.271366 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:26:14.282220 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:26:14.283640 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:26:14.292010 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1435/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:26:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:26:23.861282 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=80 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:26:24.288354 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=58 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:28:14.853479 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f01d9-18cc-454c-88e8-585272493f44 None None] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:28:14.863626 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f01d9-18cc-454c-88e8-585272493f44 None None] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:28:14.867755 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '754c137a554941d99f7ef9c380a55ca0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:28:14.868396 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:28:14.868672 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:28:14.868935 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:28:14.871187 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:28:14.935692 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:28:14.937164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:28:14.947332 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1455/2890] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:28:18.100195 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:28:18.101873 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:28:18.118553 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95367266-85e4-404d-9590-aabc2eaf0231 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:28:18.124277 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:28:18.124934 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:28:18.125272 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:28:18.125550 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:28:18.127455 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:28:18.140342 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:28:18.143870 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:28:18.157325 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1436/2891] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:28:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:28:21.120628 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:28:21.122301 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:28:21.135467 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cb693-f08a-4cc3-a1f6-8cab4ec64f07 service neutron] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:28:21.140050 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:28:21.140941 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:28:21.141251 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:28:21.141524 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] PATH_INFO: `/v3/users/5279c7bf349746b4a4622f72945662ea` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:28:21.142712 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5279c7bf349746b4a4622f72945662ea)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:28:21.144646 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:28:21.197000 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5279c7bf349746b4a4622f72945662ea'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.197670 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5279c7bf349746b4a4622f72945662ea'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.198265 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5279c7bf349746b4a4622f72945662ea'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.210006 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5279c7bf349746b4a4622f72945662ea'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.216638 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5279c7bf349746b4a4622f72945662ea'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.230925 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1456/2892] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:28:21 2022] DELETE /identity/v3/users/5279c7bf349746b4a4622f72945662ea => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:28:21.419238 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:28:21.431503 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924850a9-b1e9-4130-8b61-e708b49c2f52 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:28:21.496339 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:28:21.497242 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:28:21.497747 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:28:21.498020 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] PATH_INFO: `/v3/projects/883812495f3940a3bdeff8d4c08a29c3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:28:21.499225 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=883812495f3940a3bdeff8d4c08a29c3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:28:21.501158 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:28:21.549925 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883812495f3940a3bdeff8d4c08a29c3', 'request_id': 'req-3590c39c-5293-4aa7-960f-6c788651a610'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.559634 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883812495f3940a3bdeff8d4c08a29c3', 'request_id': 'req-3590c39c-5293-4aa7-960f-6c788651a610'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.566876 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883812495f3940a3bdeff8d4c08a29c3', 'request_id': 'req-3590c39c-5293-4aa7-960f-6c788651a610'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.579278 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] The token cache is being invalidate because project 883812495f3940a3bdeff8d4c08a29c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:28:21.579602 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.580359 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.582068 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1437/2893] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:28:21 2022] DELETE /identity/v3/projects/883812495f3940a3bdeff8d4c08a29c3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:28:21.587028 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:28:21.588730 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:28:21.602787 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b89de-e099-4693-9e02-8273971e6b53 admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:28:21.665937 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:28:21.666817 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:28:21.667099 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:28:21.667371 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] PATH_INFO: `/v3/users/6b6c519eb06a4a4cb62f38624f7f6228` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:28:21.668722 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6c519eb06a4a4cb62f38624f7f6228)` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:28:21.670734 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:28:21.676344 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:28:21.720325 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6c519eb06a4a4cb62f38624f7f6228'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.720984 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6c519eb06a4a4cb62f38624f7f6228'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.721596 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6c519eb06a4a4cb62f38624f7f6228'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.733164 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6c519eb06a4a4cb62f38624f7f6228'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.740651 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.notifications [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6c519eb06a4a4cb62f38624f7f6228'} {{(pid=80473) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:21.756165 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1457/2894] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 07:28:21 2022] DELETE /identity/v3/users/6b6c519eb06a4a4cb62f38624f7f6228 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:28:21.943485 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:28:21.945162 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:28:21.961216 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590c39c-5293-4aa7-960f-6c788651a610 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:28:22.026276 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3921c7a7e4fd41ef9eea262b7a97405a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91c604937dd14031814263870ccac629', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:28:22.027297 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:28:22.027583 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:28:22.027857 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] PATH_INFO: `/v3/projects/31ed8ccb90b1421bb77d691f395b43d5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:28:22.029204 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ed8ccb90b1421bb77d691f395b43d5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:28:22.031486 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:28:22.052793 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:28:22.079118 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ed8ccb90b1421bb77d691f395b43d5', 'request_id': 'req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:22.087258 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ed8ccb90b1421bb77d691f395b43d5', 'request_id': 'req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:22.093565 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ed8ccb90b1421bb77d691f395b43d5', 'request_id': 'req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:22.104007 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] The token cache is being invalidate because project 31ed8ccb90b1421bb77d691f395b43d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 07:28:22.104296 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:22.104927 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed2348b-afc0-4b2c-9e8a-7ef2a8ffd457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 07:28:22.106539 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1438/2895] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 07:28:21 2022] DELETE /identity/v3/projects/31ed8ccb90b1421bb77d691f395b43d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 07:28:31.753022 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:28:32.101207 dv-cinder-825492-25-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 07:31:17.209094 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Authenticating user token {{(pid=80473) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 07:31:17.210793 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:31:17.356861 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b147f-0d47-4fad-b9f9-23f4dff4331f admin admin] Validating token access rules against request {{(pid=80473) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 07:31:17.429041 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b226b1-9344-4e37-916f-4ba5b90ac2f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60ca99d3679b4154a669661e28e8269b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f18dfebf66b2442d8a63b5dd4f648f69', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80473) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 07:31:17.429702 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b226b1-9344-4e37-916f-4ba5b90ac2f4 service placement] REQUEST_METHOD: `GET` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 07:31:17.430008 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b226b1-9344-4e37-916f-4ba5b90ac2f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 07:31:17.430285 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b226b1-9344-4e37-916f-4ba5b90ac2f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80473) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 07:31:17.432476 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.fernet_utils [None req-b4b226b1-9344-4e37-916f-4ba5b90ac2f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80473) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 07:31:17.444848 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b226b1-9344-4e37-916f-4ba5b90ac2f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 07:31:17.446525 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b226b1-9344-4e37-916f-4ba5b90ac2f4 service placement] RBAC: Authorization granted {{(pid=80473) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 07:31:17.458108 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: [pid: 80473|app: 0|req: 1458/2896] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 07:31:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 07:31:27.453034 dv-cinder-825492-25-iscsi devstack@keystone.service[80473]: DEBUG dbcounter [-] [80473] Writing DB stats keystone:SELECT=89 {{(pid=80473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}